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Advancing the application of systems thinking in 


health 


Taghreed Adam 


Editorial 

Highlighting the growing significance of systems thinking 
in health: introducing a new global series 

The past two decades have witnessed an increased recog- 
nition among global health stakeholders of the importance 
of systematically considering the complex adaptive nature 
of health systems to better anticipate some of the unex- 
pected and counterintuitive consequences of implement- 
ing current and new policies. This is evidenced by the 
increased interest in topics such as systems thinking, com- 
plex adaptive systems, and systems science in the pub- 
lished health literature over the past 20 years (Figure 1). 
However, the majority of these publications are from 
high-income countries, while the need for applying these 
concepts is at least as great in low- and middle-income 
countries (LMICs) (Figure 2). Most of these studies dis- 
cuss the concepts or make the case for the utility of sys- 
tems thinking for health systems strengthening; there is 
still a dearth in practical guidance on how systems think- 
ing concepts, approaches, and tools can be applied in 
health systems research and practice to reach sustainable 
solutions [1,2]. 

Systems thinking is, foremost, a mindset that views sys- 
tems and their sub-components as intimately interrelated 
and connected to each other, believing that mastering our 
understanding of how things work lies in interpreting 
interrelationships and interactions within and between 
systems [1,3,4]. It is a perspective that deliberately goes 
beyond events, to look for patterns of behavior and the 
underlying systemic interrelationships which are respon- 
sible for these patterns and their associated events [5]. It 
embraces the understanding of open systems as complex 
adaptive systems that are constantly changing, resistant to 
change, counter-intuitive, non-linear, and where the whole 
is greater than the sum of the parts [3]. 

The Alliance for Health Policy and Systems Research 
(hereafter called the Alliance) has been one of the avid 
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advocates for moving this kind of thinking forward, dedi- 
cating a number of activities and resources to promote 
this field among health practitioners and researchers. First, 
through its flagship publication on “Systems Thinking for 
Health Systems Strengthening” in 2009 [5], followed by a 
Journal Supplement in Health Policy and Planning, in 
2012, it has sought to generate better understanding of 
current practices in applying systems thinking for health 
systems in LMICs [1]. 

The 2012 supplement demonstrated the dearth of appli- 
cations that explicitly took into account the complexity 
and dynamics resulting from intervening in health sys- 
tems, including evaluations of interventions with system- 
wide effects [2]. In addition, the very few applications that 
existed at the time of developing that supplement were 
predominately from high income countries [1]. These 
observations revealed the need for concerted efforts to 
advance the application of systems thinking in health, 
particularly in LMICs. 

In March 2013, the Alliance, in collaboration with 
Canada’s International Development Research Centre, 
launched a Call for papers inviting teams of researchers 
and health practitioners, with particular focus on lead 
authorship from LMICs, to develop and share applica- 
tions of systems thinking methods and approaches, cul- 
minating in this Series. This whole program of work, 
which spanned over two and a half years, provided a 
great opportunity for strengthening programs, policies, 
and methods in LMICs to enable researchers and deci- 
sion makers to think through how systems thinking ap- 
proaches can be applied to their current health systems 
questions with practical results. 

It is worth noting that, while this collection of articles 
offers innovative and diverse range of applications of 
systems thinking approaches, methods, and tools, as the 
Commentary by Peters illustrates [6], these applications 
by no means capture the entire range of relevant tools 
and approaches that can be applied. 
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Figure 1 Trends in the use of the terms “systems thinking”, “complex adaptive systems”, or “systems science” in the Medline database 
over the past 40 years. Source: GoPubMed, which reports the frequency that terrns appear in MEDLINE indexes for publications, which include titles, 
abstracts, journal names and corresponding author's affiliation. Number of publications mentioning these search terms was 1386 as of 14 August 2014. 


The applicability of a wide range of tools and approaches 
This Series illustrates how research approaches that are 
commonly used in various disciplines, such as realist 
evaluation, sense-making (as a mental model), or program 
evaluation theories, can be applied within a systems think- 
ing approach to address complex health systems ques- 
tions. It does so by showing that the types of questions 
asked are the most important element that shape the 
orientation of the analysis, not the tool itself. 

For example, in the paper by Prashanth et al. [7], SyS- 
tems thinking and complex adaptive systems approaches 
added depth to the realist evaluation by digging deeper 
into the drivers of, and the context in which the differ- 
ences in responses of health workers in the two sub- 
districts were observed and what triggered them. They 
could show that settings with committed staff and po- 
sitive intentions to make changes demonstrated more 
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positive outcomes and an ability to use existing oppor- 
tunities to solve problems and improve performance. 
Further, that commitment alone was neither crucial nor 
sufficient as demonstrated by findings from another 
setting with committed staff but different outcomes. Fi- 
nally, that in settings with a lack of commitment from 
staff, strong leadership became more pronounced in 
driving the change into better outcomes [7]. 


Systems thinking and mixed methods 

As discussed by Peters and demonstrated by several of 
the Series papers, both qualitative and quantitative 
methods contribute in their own way to our understand- 
ing of complexity [6,8]. As some of the early systems 
thinking literature originated from quantitative disci- 
plines such as physics and biology, it may give the im- 
pression that relevant systems thinking approaches are 
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predominantly quantitative. Perhaps one of the main 
contributions of this Series is demonstrating how quali- 
tative methods commonly used in fields such as social 
science or anthropology add equally important value and 
depth to analyses of complex health systems questions 
and phenomena [8-12]. For example, they are often used 
to provide a profound initial understanding of the prob- 
lem that can then be complemented by quantitative ap- 
proaches that incorporate the learning into a more 
realistic and sophisticated quantitative analysis [6]. 


Exploiting the potential of visual interpretations of 
complex phenomena 

During the past decade there has been a revolution of 
infographics due to the increased recognition of the 
power of graphics to aid data interpretation and decision 
making. In this Series, several papers illustrate how a 
range of graphic tools can help convey complex inter- 
pretations and findings in a meaningful visual form, 
namely causal loop diagrams [8-12], stakeholder network 
analysis, and sociographs [13,14]. 

For example, causal loop diagrams used by Rwashana 
et al. to understand the causes of neonatal mortality 
in Uganda not only helped analyze and make sense of 
the different sources of data in a dynamic and itera- 
tive way, they were also used to present these com- 
plex findings in one main diagram that summarized 
the relationships, dynamics, and associated factors all 
in one graph [8]. 

Another example of visual interpretations is presented 
in Malik et al., where they used sociographs to interpret 
the pattern of advice-seeking behavior among primary 
health care physicians and the potential explanations for 
their choices [13]. 


Content of the series 

The Series covers a range of systems thinking methods, 
tools, and approaches, including system dynamics model- 
ing [15], causal loop diagrams [8-12], and social network 
analysis [13,14]. In addition, several papers couched 
their analysis in a complex adaptive systems framework 
(9,10,16], or adapted established methods, such as real- 
ist evaluation [7,12] and policy analysis [16,17], to un- 
tangle the underlying complexity of their research 
questions. The main approaches, research questions. 
and findings of the Series papers are discussed in turn 
below. 

The paper by Bishai et al. uses a system dynamics 
simulation model to illustrate trade-offs and unintended 
consequences in allocative funding decisions to curative 
versus preventive care [15]. The model provides a quan- 
titative application of complex adaptive systems method- 
ologies to a health systems and policy question, something 
that traditional cost-effectiveness analysis techniques fail 
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to illustrate. In this paper, the authors demonstrate how 
the growth of curative care services can crowd both fiscal 
space and policy space for delivering population-level pre- 
vention services, which would require extensive and long- 
term interventions to overcome the fiscal and population 
health consequences [15]. 

The paper by Prashanth et al. is one of three papers 
exploring capacity strengthening initiatives targeting 
health workers and managers [7]. They use realist evalu- 
ation to explore how a capacity building intervention for 
district health managers implemented in two different 
places evolved over time, taking into account the context 
and the mechanism of change. The paper highlights the 
importance of the people involved and the choices they 
make in the evolution of outcomes, and how individual 
and organizational attributes and the interaction be- 
tween them contribute to any particular outcome [7]. 

Kwamie et al. is another paper looking at capacity 
strengthening of middle-range managers [12]. The au- 
thors also used realist evaluation, supplemented by visu- 
ally interpreting their findings using a causal loop 
diagram, to examine how and why a Leadership Devel- 
opment Programme works when it is introduced into a 
district health system and whether or not it supports 
systems thinking in district teams, using Ghana as a case 
study. They conclude that the leadership program on its 
own did not lead to the development of a systems thinking 
approach in management and decision-making in the dis- 
trict and argue that the complexity of organizational con- 
texts and history are important influencing factors for the 
sustainability or scaling up of such programs, as much as 
the complexity of the intervention itself [12]. 

Gilson et al. stimulate wider thinking about the forms 
and practices of health leadership [17]. They use the con- 
cepts of sense-making and discretionary power drawn from 
the theories of complex adaptive systems and policy imple- 
mentation to highlight how important it is that health sys- 
tem actors are able to make sense of the intentions of 
policies to be able to incorporate them into their everyday 
routines and practices. The study reveals how the collective 
staff understanding of their working environment, and 
how changes occur within it, may act as a barrier to 
centrally-led initiatives to strengthen capacities [17]. 

Next, in an application of social network analysis, the 
study by Malik et al. describes the formal and informal 
ways in which primary care physicians in Pakistan access 
information [13]. By employing a range of research 
methods, the paper examines the reasons for the dispar- 
ity between organizational structures for supervisory and 
reporting relationships and the actual behavior of pri- 
mary care physicians when seeking information. They 
argue for the importance and value of exploring the 
supervisory and technical support arrangements from 
the view point of the users [13]. 
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In the paper by Paina et al. [9], the authors investi- 
gate how, in a context of no official government po- 
licy on dual practice, this practice is currently being 
regulated in Uganda through a system of unwritten 
expectations and local management practices that have 
not been elsewhere documented. Through a series of 
causal loop diagrams and historical and primary ac- 
counts, the authors depict the resulting behavioral pat- 
terns and complex systems characteristics such as policy 
resistance in the form of protests by government pro- 
viders and coping approaches by providers and their 
managers to maintain public sector’s service delivery 
and performance [9]. 

Rwashana et al. offer another application of causal loop 
diagrams in exploring the complexity which characterize 
neonatal health and its interplay with health systems fac- 
tors, using Uganda as a case study [8]. The analysis re- 
vealed multiple feedback loops, such as trust, that 
household place on the health system, awareness of the 
benefits of antenatal and postnatal care, myths, frustration 
of health workers and its impact on all aspects of their 
performance, among others. The authors also discuss high 
leverage points that may be considered by policy makers 
to improve neonatal health such as gender considerations 
related to girls education and empowerment [8]. 

Next, in their analysis of the causes of reductions in 
coverage of vaccination in Kerala, Varghese et al. dem- 
onstrate how important it is that the evidence used to 
design and evaluate public health programs goes beyond 
epidemiological and economic analysis [10]. The paper 
shows how key factors that contributed to the unex- 
pected decline in vaccination coverage were revealed 
such as how the opposition by the government medical 
doctors association and alternative medicines propo- 
nents, compounded by strong media influence, have 
evolved overtime and created a big unexpected influence 
on people's decision to vaccinate [10]. 

The following two papers explore experiences with two 
financing schemes in Ghana and China. In their ana- 
lysis of the Ghana National Health Insurance Scheme, 
Agyepong et al. suggest that relatively less attention seems 
to be paid to service access and service responsiveness 
when evaluating an insurance scheme [11]; something that 
people thinking about enrolling consider as much as 
the financial risk protection potential. They highlight 
why a comprehensive systems thinking approach is es- 
sential when conceptualization and designing a health 
insurance scheme to avoid the emergence of counterin- 
tuitive and undesired effects [11]. Zhang et al. offer an- 
other perspective of a financing intervention, by exploring 
the evolution of rural finance schemes in China [16]. The 
paper discusses the nature of health systems resilience fa- 
cing the implementation problems associated with the 
policy and argues that initial trajectories have been a big 
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determinant of how policy-makers adapted in the various 
contexts [16]. 

Blanchet et al. examine sustainability in internationally- 
initiated programs in their comparison of sustainability- 
oriented processes in two rehabilitation centres in Nepal 
and Somaliland [14]. The paper shows how differences in 
the governance and network structure of the rehabilitation 
centres, revealed through a stakeholder network analysis, 
have influenced the process and commitment to sustain- 
ability. The analysis helped in the understanding of change 
in the nature of relationships between actors and their 
capacity to work together over time and what factors are 
conducive to providing the right incentives to work as an 
interrelated network rather than as individual actors [14]. 

In second paper addressing sustainability issues, Sar- 
riot et al. examine how an international NGO worked 
with two Bangladeshi municipal health departments to 
intentionally advance sustainability in their support for 
maternal and child health preventive services [18]. The 
paper explores how systems thinking was used to gener- 
ate a process of change within municipal health systems, 
affecting technical, social, political, and organizational 
sub-systems. The authors document how a sustainability 
framework method was used to work with stakeholders 
in an explicit process to guide their decisions and choices 
during and after the life of a project. They illustrate how 
this process offered useful tools to engage stakeholders, 
give shared meaning to information about activities and 
achievements, facilitate decision making, and mitigate the 
risk of unintended project effects in order to achieve a 
measure of sustainability in a complex setting [18]. 

Last but not least, the commentary by Peters discusses 
which of the large body of theories, tools, and methods 
associated with systems thinking are more useful to un- 
derstanding the behaviour and complexity of health sys- 
tems [6]. It also discusses the “jungle” of terminology 
surrounding this field and how and why some terms 
have emerged and been used differently in different dis- 
ciplines. It then provides a helpful overview of a wide 
range of systems thinking theories, methods, and tools 
that are relevant to understanding and exploring health 
systems questions [6]. 


Looking forward 

With the selection of papers in this Series, our aim was to 
give meaning to abstract concepts and theories through 
actual applications and experiences of how systems think- 
ing tools and concepts can be used to understand and 
strengthen health systems, particularly in LMICs. We 
hope that by providing a variety of experiences, examples, 
and ideas that are relevant to other complex interventions 
and contexts, this collection will stimulate wider applica- 
tions and innovations of these and other approaches rele- 
vant to this field. 
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Abstract 


accumulated revenue. 


by ring-fencing it to protect it from lobbying. 


funding. 


interventions to overcome these trends. 


Introduction 

Achieving optimum health of a population requires an art- 
ful combination of preventing ill health and responding to 
disease cases with curative services. Both are important, 
but there are predictable obstacles to achieving balance. 
Too often, only a single, limited government health 
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Introduction: This paper presents a system dynamics computer simulation model to illustrate unintended 
consequences of apparently rational allocations to curative and preventive services. 


Methods: A modeled population is subject to only two diseases. Disease A is a curable disease that can be 
shortened by curative care. Disease B is an instantly fatal but preventable disease. Curative care workers are 
financed by public spending and private fees to cure disease A. Non-personal, preventive services are delivered 
by public health workers supported solely by public spending to prevent disease B. Each type of worker tries to tilt the 
balance of government spending towards their interests. Their influence on the government is proportional to their 


Results: The model demonstrates effects on lost disability-adjusted life years and costs over the course of several 
epidemics of each disease. Policy interventions are tested including; i) an outside donor rationally donates extra 
money to each type of disease precisely in proportion to the size of epidemics of each disease: ii) lobbying is eliminated; 
iii) fees for personal health services are eliminated; iv) the government continually rebalances the funding for prevention 


The model exhibits a “spend more get less” equilibrium in which higher revenue by the curative sector is used to 
influence government allocations away from prevention towards cure. Spending more on curing disease A leads 
paradoxically to a higher overall disease burden of unprevented cases of disease B. This paradoxical behavior of the 
model can be stopped by eliminating lobbying, eliminating fees for curative services, and ring-fencing public health 


Conclusions: We have created an artificial system as a laboratory to gain insights about the trade-offs between curative 
and preventive health allocations, and the effect of indicative policy interventions. The underlying dynamics of this 
artificial system resemble features of modern health systems where a self-perpetuating industry has grown up around 
disease-specific curative programs like HIV/AIDS or malaria. The model shows how the growth of curative care services 
can crowd both fiscal and policy space for the practice of population level prevention work, requiring dramatic 
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budget is available for investments in both non-personal 
preventive and curative personal health services. A variety 
of influences affect the allocation of this common health 


budget [1]. Ideally, these factors include efficiency and 
equity. In reality, concerns also include policy and political 
priorities, which often take precedence over efficiency and 
equity criteria [2]. This struggle between evidence-based 
decision-making to achieve health system goals and the 
reality of policy and financing constraints occurs in a 
variety of contexts besides government health ministries. 
Fixed government health budgets lend themselves to a 
zero-sum game in resource allocation between cure and 
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prevention. More spending on curing diseases will mean 
less for prevention and vice versa. It is observed with re- 
gularity, in both high-income and low- and middle- 
income settings, that whenever there is a fixed sum to 
be allocated between curing and preventing diseases, a 
higher total will be spent on curing than preventing, and 
more will be spent per disability-adjusted life year 
(DALY) averted by curing than preventing [3-6]. For 
example, although the burden of disease associated 
with chronic, non-communicable diseases is significant, in 
the Organization for Economic Co-operation and Devel- 
opment countries average expenditure on public health 
and prevention for non-communicable diseases was only 
3% of the total health expenditure in 2005, while average 
expenditure on curative care was 57% [7]. The situation 
can be even direr in developing countries, particularly in 
sub-Saharan Africa, where the large urban hospitals often 
receive at least half of the public funds spent on health [8]. 

Spending more money per DALY averted on curing 
than preventing violates both efficiency and equity goals. It 
violates efficiency standards by definition. If intervention P 
saves more lives per incremental dollar than intervention 
C then a shift of spending from C to P will save more lives 
but it will not cost more. It also violates equity standards 
because access to curative services is often achieved pref- 
erentially by those with greater social privilege [9]. The 
preventive interventions we consider here are delivered to 
a population en masse rather than in individual clinical 
encounters and they have been shown to decrease popula- 
tion health disparities and increase health equity [10,11]. 

Preventing is not always more efficient than curing. 
Many preventive health care procedures delivered to in- 
dividuals in clinics are not uniformly more cost-effective 
than curative clinical services [12]. However, most pre- 
ventive interventions are not clinical procedures, they are 
community and environmental interventions mounted by 
public health entities. By shifting the environmental and 
social determinants of health for populations millions at a 
time, public health expenditures are typically best buys in 
health [13]. 

This paper analyzes a process whereby a neutral policy 
change undertaken in the name of efficiency can lead 
to a spiraling increase in the power of groups whose 
self-interest will block rational and efficient allocation 
of public resources in the future. A standard decision 
analysis of option A vs. option B will be inadequate if 
option A commits future generations to depart from 
rational policy making because of the power of interest 
groups created by option A. To be specific, the model de- 
veloped here examines how health policies may enhance 
the class power of curative care interests (e.g., clinicians, 
hospitals, medicine manufacturers) and lead to a snowball 
effect that exaggerates a bias to spend more towards 
clinically-oriented health spending in the future. 
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Power politics are unavoidable in health policy [14]. 
Examples of policy makers successfully appealing to 
cost-effectiveness data and not politics to rationalize 
their health spending portfolio are few [15]. In fact, 
many examples show that policy-makers do not use 
cost-effectiveness data to decide on budget allocation 
[16,17]. There are simple explanations for why decision- 
makers whose stated objective in budgetary allocation is 
to avert DALYs at lowest cost fail to actually allocate 
spending accordingly. Most explanations focus on the 
decision-makers lack of cost-effectiveness data or un- 
familiarity with the paradigm [2,12]. However, the regu- 
larity of the bias towards cure and away from prevention 
suggests that something more structural and systematic 
must be at work. 

The models that will yield understanding will need to 
encompass unintended consequences of complex adap- 
tive systems. A growing body of literature explores the 
role of complexity in health systems [18-21]. For this 
paper, we use the principles of system dynamics modeling 
to develop understanding of non-linear interactions in de- 
fined systems [22]. Using system dynamics, researchers 
can simulate policy scenarios which cannot be carried out 
in real populations or for which adequate historical data 
on natural experiments is not available. We offer a simple 
model of political lobbying between the curative health 
care sector and public health proponents situated in a 
hypothetical population with a very simple epidemio- 
logical problem. 

We are pursuing a form of “generative” social science— 
applying the adage “if you didn't grow it, you didn’t explain 
it” [23]; being able to generate a phenomenon without 
pre-supposing it is the best way to understand it, and this 
is best done in a simulation. The model is simple enough 
so that we can turn parts on and off allowing readers to 
understand which of the dynamics emerge from the 
simplistic assumptions and which result from the policy 
experiments. The omitted complexities of real world spend- 
ing and lobbying would certainly mitigate the dire con- 
sequences that befall the population in our simplified 
model. That underscores the advantage of the model in 
offering insight into processes that are harder to measure 
in the thicket of real world observations. The model 
intentionally exaggerates important aspects of the real 
system — the exaggeration is a feature, not a flaw. 

This artificial model is not trying to fit any real-world 
epidemiological data. The purpose of the model is to 
gain understanding of elementary political forces that 
can be turned on and off in the model. Specifically, one 
can test the effect on population health of government 
policies that: 


i) Rationally allocate spending according to neutral 
cost-effectiveness criteria. 
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ii) Allocate government funds proportionally to interest 
group lobbying or not. 

iii) Accept or refuse spending by non-governmental 
organization (NGO) external donors that target 
either disease A or B. 

iv) Allow fee-for-service revenue for curative care 


workers. 


Methods 

We model a finite population that is susceptible to only 
two possible unwanted health conditions. Individuals can 
suffer from an acute disease, called disease A, that is disab- 
ling, never lethal, and whose duration can be shortened by 
a visit to a doctor. They can also suffer from a sudden, in- 
stantaneously lethal condition, disease B, that can never be 
cured, but which can be prevented through environmental 
engineering by preventive care workers (PCWs) (if it helps, 
one can think of disease A as something like an intestinal 
parasite, e.g., ascariasis, and disease B as something like 
a bicycle crash). Disease A, if untreated, causes 100% 
disability for 0.5 years before recovery. Each untreated 
case of disease A imposes 0.5 DALYs on the population. 
Disease B kills each victim instantly and deprives them 
of 25 additional years of survival; each case of disease B 
imposes 25 DALYS. Both the doctors and the PCWs are 
supported by a fixed-sum budget allocated by the gov- 
ernment. In contrast to the PCWs, the doctors can 
also collect fees from each patient who sees them after 
contracting disease A. Both PCWs and doctors invest 
a similar portion of their earnings on lobbying the 
government for a larger share of the fixed-sum budget. 
The government can be swayed by the lobbying, in 
which case it allocates the budget proportionally to the 
size of the lobbying funds of the respective two parties, 
e.g., if doctors account for 2/3 of all of the money spent 
on lobbying, they will get 2/3 of the health budget. 

In the baseline mode, initial values were selected to 
put the model at a stable equilibrium prior to the intro- 
duction of shocks to the system. The initial baseline 
equilibrium does not include any external funding, and 
there are no shifts in incidence of either disease. In the 
baseline, all inflow and outflow equations are balanced 
perfectly. Subsequently, the response of the model to 
policy shocks can be fully attributed to the policy changes. 
A stock-and-flow diagram and simulation model was cre- 
ated using the VensimPLE® software [24]. Consistent with 
system dynamics methodology, there are three types of 
variables for each sub-system: state variables depicting 
levels, difference equations depicting flow rates, and 
auxiliary variables reflecting other parameters 22h. 
Our stock-and-flow model is comprised of the three 
subsystems described below. 

The model was designed solely by the investigators 
during a number of team meetings by DB, LP, and QL to 
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redraw and recalibrate the feedback loops and adjust the 
parameters based on model output. Some studies in sys- 
tems science are done with a specific decision-maker or 
institutional client in mind. In these cases, it is quite 
common to involve those decision-makers in helping to 
design the model. This engages the community of practice 
and research together and helps the group interpret the 
output of the model to jointly improve organizational pol- 
icy. However, a drawback of this bespoke approach is that 
the better the model fits a specific problem, the less it fits 
a general problem. The research here is intended to be of 
general relevance to any setting where there is a zero sum 
budget that could be allocated to prevention or cure. 


Subsystem 1: the population and disease model 

The subsystem for the population and its experience of 
diseases A and B can be found in Figure 1. Table 1 presents 
the population parameters, initial values, and relevant 
assumptions. There is a stable healthy population in 
which a person stops being healthy either temporarily 
if an individual contracts disease A, from which each 
will recover after a certain duration determined by the 
activity of doctors. Population members can also exit 
permanently by sudden death from disease B. If they 
do not die of disease B, individuals will all die 25 years 
later through a process unspecified by the model. The 
population count remains stable because fertility is unim- 
portant to the focus of the model. All deaths are immedi- 
ately replaced by new healthy full grown individuals with 
remaining lifespans of 25 years. The duration of disease A 
is influenced by medical care expenditures by government 
(and donors) and by fees paid to doctors in exchange for 
curative services. Both the acquisition of disease A and 
death from disease B lead to increments in the DALYS 
lost by the population. Hence, the relative impact of one 
event of either condition on the DALY measure of popula- 
tion health is just a matter of the arbitrary DALY weights. 
In the baseline model, we set these weights so that the 
population experiences an equal burden of DALYS from 
diseases A and B. 

The fundamental difference between the curable dis- 
ease A and the preventable disease B in the model is not 
the DALY weights assigned to each as later shown in the 
sensitivity analysis. The key asymmetry between A and B 
is that there is no market for PCWs to charge fees for 
preventing disease B. Epidemics of curable diseases will 
fill the doctor’s offices with paying customers, but an 
epidemic of disease B cannot generate a revenue surge 
in the absence of government action. This difference in 
the model corresponds to a difference in real health policy, 
especially remembering that the “prevention” that is being 
modeled is not clinical preventive services which can 
generate revenue, but community level public health 
activities (e.g., road hazard reduction) for which revenue 
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Figure 1 Subsystem 1: the population model. A susceptible population is at risk for either dying from disease B or transitioning temporarily 


of A 
wait YR A> 
| ; Po et 
A Annual DB A Acumm DB 
B Acums DB 
| | A maence 5 Anse DB 
ie, 
| i 
i <¥h. Be 
| | 
| | 
| 5 Deaths 
| 
: 
| 


into disease A. Abbreviations: A Acumu DB, Accumulated DALYS from A; B Acumu DB, Accumulated DALYs from B; Total Accumu DB, Total 
| accumulated DALYs. 


collection is necessarily collective. Even though real world equal baseline incomes helps us see exactly what is re- 
doctors have higher incomes than PCWs, we set their sponsible for the power imbalances that will emerge. 
baseline income to be equal in the model. The incomes The power imbalance between cure and prevention is 
become unbalanced only when we allow the doctors to not assumed — the imbalance comes from policies that 
charge fees to people with disease A. The assumption of _ tie earning to curing. 


Table 1 Population subsystem parameters and assumptions 


Equilibrium parameters Variable type Initial value Notes and assumptions 

Susceptible population State 800 a Individuals = 

Disease A infection rate Rate 40 Individual infected/year 

Disease A patients State 200 Infected population, at equilibrium 

Disease A incidence rate Auxiliary 0.05 Population based incidence 

Disease A duration Auxiliary 0.2 Years 

Disease A recovery rate Rate 40 All infected with disease A, recover 

Death from disease B Rate 4 Deaths/year 

Accumulated death from disease B State 0 Deaths from B, at equilibrium 

Disease A annual burden Auxiliary 100 Disease A burden at equilibrium 

DALYs lost per case of disease A Auxiliary 0.5 Constant — one untreated case of infection with 
A is equivalent to 0.5 years of life with disability 

Disease A cumulative burden State 0 Disease A burden at equilibrium 

Disease B annual burden Auxiliary 100 Disease B burden at equilibrium 

DALYs lost per case of disease B Auxiliary 25 Constant — one death from B is equivalent to 
25 years of life lost 

Disease B cumulative burden State 0 Disease B burden at equilibrium 

Total curnulative disease burden Auxiliary 0 Sum of cumulative burden for disease A and 


disease B, at equilibrium 
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Subsystem 2: health resources 
The subsystem depicting health resources and how they 


are allocated is illustrated in Figure 2. Table 2 presents 
parameters, their units, and any assumptions that are 
relevant for our design. The simulated health system has 
three principal funding sources: public funding for both 
PCWs and doctors, private spending for curative services 
only, and donor funding. Public funding is a finite source 
which is allocated between doctors and PCWs through a 
political bargaining process between the two categories of 
health workers vying for a finite pot of governmental 
health funds. Bargaining power is measured in monetary 
units and each group acquires “power” by paying a portion 
of their earnings into a bargaining fund. In a simple model 
of “rent-seeking” the government is influenced to allocate 
state resources to either group in proportion to their share 
of total bargaining power. Thus, allocation of the fixed 
health budget to C is proportional to C/(C + P) and alloca- 
tion to P follows P/(C +P) where C is lobbying dollars 
spent by curative care interests and P is lobbying dollars 
spent by preventive care interests. 

The model sets initial values such that the baseline 
government spending is economically optimal. At base- 
line, the dollars the government spends per DALY averted 
is exactly the same between diseases A and B. Emotional 
or political factors that might bias spending towards cure 
are not burnt in as an official part of government strategy. 
In addition to public funding, we set initial values so that 
the arrival of external aid from donors is always unbiased 
making the donor allocate new dollars to disease A or dis- 
ease B to keep the incremental DALYS averted per dollar 
equal and unbiased. Such a policy is crudely feasible for 
donors today who could consult league tables that display 
comparative $ per DALY averted from various inter- 
ventions across world regions [13,25]. In the baseline 
version of the model, these efficient spending levels are 
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maintained in perpetuity because there are no epidemics 
that might trigger donor allocations or patient care seek- 
ing. Lobbying power and funding allocations between doc- 
tors and PCWs will remain perfectly balanced as long as 
incidence rates of either disease are unperturbed. 

For policy simulation, the model imposes an exogenous 
series of epidemics of A and B as a series of step functions 
that raise the incidence of diseases A or B or both above 
the equilibrium every 3 to 4 years. The model is able to 
restore equilibrium after these step function epidemics 
because the government and/or donors immediately 
detect the epidemic and they rationally increase funding 
towards whatever disease has risen above its baseline. The 
government and donors are programmed to respond with- 
out bias to disease A or B. They are set to allocate the 
same $ per DALY averted to disease A as is allocated to 
disease B during an epidemic, as long as there is no polit- 
ical lobbying. 


Subsystem 3: doctor and PCW resource allocation 

Figures 3 and 4 illustrate the final subsystem in the 
model — the lobbying process driven by doctors and 
PCWs. Table 3 presents the parameters, their units, and 
any assumptions that are relevant. We define political 
bargaining power of doctors as proportional to resources 
collected in a common lobbying fund by doctors out of 
their public and private salaries. The doctors, as a group, 
can then allocate the money they earn from fees, govern- 
ment payments, and donor payments to either improving 
recovery time for disease A in the infected population or 
to lobbying activity so they can capture more public funds. 
Similarly, we define PCW power as the resources amassed 
by PCWs from all sources, which can then be allocated to 
preventive services or lobbying. As mentioned above, PCW 
resources include public funds and donor funding, but there 
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Table 2 Subsystem 2 variables and assumptions 

Equilibrium parameters: ane Variable type : oi 4 Initial value Notes and assumptions 

Public fund Auxiliary 1250 Monetary units available, at equilibrium ¢ 

Political influence State N/A (Doctors spending on lobbying)/(Doctors spending on 
lobbying + PCWs spending on lobbying) 

Public pay to doctors Rate N/A (Political Influence*Public Fund) + NGO for doctor 

Private pay for doctors Rate N/A 

Consultation fees for doctors Auxiliary 3.75 Monetary units per treatment provided by doctor 

Public pay to PCWs Rate N/A Public Fund x (1—Political Influence) + NGO for PCW 

NGO-D State 0 Accumulated external funding to doctors 

NGO-P A State 0 Accumulated external funding to PCWs 


are no fees paid for PCWs. The entire model, showing how 
the sub-systems relate, is displayed in Figure 5. 


Modeling scenarios 
The model horizon is 200 months. During both model- 
ing and sensitivity analyses, we tracked changes in cu- 
mulative disease burden and deaths from both diseases 
and cumulative spending. We used the cumulative bur- 
den of disease and the cumulative spending to produce 
the graphs of costs vs. DALYs averted by the various 
donor spending policies. 

A standard run with the simulation requires observing 
its response to two types of simulated shocks: disease 
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Figure 3 Subsystem 3: lobbying process ~ doctor power can be depleted by spending either on providing patient services or on 
lobbying for more resources. The proportion of power spent on lobbying is a constant fraction in each run of the model. The more doctors 


spend on patient services the shorter the duration of disease A for patients who contract that disease. 


epidemics and external funding from donors. Without 
shocks, the model stays in perpetual equilibrium. Epi- 
demic times for disease A were arbitrarily selected to fall at 
months 13, 49, 85, and 121; each epidemic lasted one year. 
During an epidemic of disease A, the incidence rate 
increased ten-fold, from 0.05 to 0.5. Epidemic times for 
disease B were selected to fall at months 37, 61, 109, and 
121. We tried different epidemic combinations and tim- 
ings. Specifically, the system encountered an epidemic of A 
alone, then an epidemic of B alone, then two epidemics of 
A followed by B, and, finally, an epidemic of A and B to- 
gether. We examined the system’s ability to contain these 
epidemics with and without the response of NGO donors. 
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Figure 4 Subsystem 3: lobbying process - PCW power is spent on lobbying and on shortening the incidence of death from disease B. 
the proportion of power spent on lobbying is a constant fraction in each run of the model, J 


NGO donors are programmed to make donations to donation policies become discordant and we tested dona- 
their chosen disease only as long as the incidence rate is tion policies with Da set sequentially at {0, 5, 10, 20, 30, 
higher than the equilibrium baseline level. Donation 40, 50, 60, 70} while, independently, Dg realized values 
policies are specified as the number of additional dollars from {20, 30, 40, 50, 60, 70} [27]. A donor whose policy 
allocated to either disease per DALY averted. Da was was Da =70 and Dg=10 would be indicating a bias to 
defined as the donation in terms of $ per additional spend seven times as much averting a DALY from dis- 
DALY of disease A after an epidemic of A and Dg asthe ease A relative to disease B. A total of 81 (Da, Dg) pairs 
donation in terms of $ per additional DALY of disease B were tested and we tracked the 200 month cumulative 
that the NGO would spend. Suppose the donation policy sum of DALYS from both of the diseases and total costs 
is Da = $10 per epidemic DALY of A and $Dz =5 per epi- to the government, patients, and NGOs as a result of the 
demic DALY of B. In an epidemic with 10 additional NGO's epidemic control strategies. Overall, 81 cycles were 
DALYs of A and 6 additional DALYs of B, the donations required through the 200 month trajectory of the model 
would be $100 to A and $30 to B. In equilibrium, D4 was to fully assess a policy or parameter change. The graphical 
set to equal Dy. There was no built-in role for donors to output of these policy assessments involves showing a 
practice disease exceptionalism [26]. To test the effects of four-dimensional manifold of costs and DALYS as co- 
special pleading on behalf of either disease, we later let determined outcomes of Da and Dy. We accomplished 


Table 3 Subsystem 3 variables and assumptions 


Equilibrium parameters Variable type Initial value Notes and assumptions 


Doctor power State 1,000 Doctor's lobbying power at equilibrium: equals PCW 
power at equilibrium 


Doctor spending on lobbying Rate 0 Doctor power x Proportion on lobbying 

Proportion on lobbying of doctors Auxiliary 0.125 Doctors spent 12.5% of their resources on lobbying 

Doctor spending on patient services Rate 0 Doctor power x (1—Proportion on lobbying of doctors) 

PCW power State 1,000 PCW's lobbying power at equilibrium; PCWs and doctors 
start with the same level of power 

PCW spending on lobbying Rate 0 PCW power x Proportion on lobbying of PCWs 

Proportion on lobbying of PCWs Auxiliary 0.5 PCWs spend 50% of their resources on lobbying 

PCWs spending on prevention of disease B Rate 0 


PCW power x (1-Proportion on lobbying of PCWs) 


—— a 
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Figure 5 System dynamic model showing the factors affecting resource allocation. This figure displays how all of the previously introduced 


sub-systems relate, 


this by showing isoquants — a locus of DALY and cost out- 
comes generated when D, (or Dx) is held constant while 
Dg (or Da) varies along the locus (Figures 6 and 7). 

The economic problem for the NGO ought to be 
choosing epidemic response policy (Da, Dp) to minimize 
DALYs while minimizing total cost to itself and to society. 
Whether or not it espouses an economic paradigm, the 
NGO does not want to achieve a situation where they 
spend more and achieve more overall deaths and DALYs 
due to their spending crowding out other activities in the 
health system. Locating attractive and unattractive values 
of (Da, Dg) is our schematic version of why donors care 
about data on the global burden of disease and why they 
want data on the effectiveness and cost-effectiveness of 
health interventions. 


Sensitivity analyses 

We conducted the following sensitivity analyses in order 
to ensure that the results we obtained from the main 
model were robust to changes in the original parameters 
and variables: 


¢ We let years of life lost from disability per 
case of disease A vary +10% from 0.45, 0.5, 
and 0.55. 

e We let years of life lost from death per case of 
disease B vary +10% from 22.5, 25, and 27.5. 


Policy analyses 

We implemented three different policy scenarios to dis- 
cover how various strategies might make the system work 
better. 


e Eliminate lobbying for curative care and preventive 
care. 

e Making the public pay responsive to burden of 
disease changes resulting from epidemics. 

e Removing the consultation fee obtained by doctors. 


Results 

We found that several negative feedback loops stabilized 
key levels in the model. There was negative feedback re- 
storing physician incomes to baseline after their incomes 
rose in an epidemic. Essentially, the epidemic gave the 
doctors more money, but they ultimately spent some of 
the money reducing the size of the caseload that generated 
their revenue and eventually their revenue fell. The bur- 
den of disease A was also restored to equilibrium through 
this negative feedback loop. 

The model also showed the presence of a negative feed- 
back loop for government to respond to epidemics of dis- 
ease B with extra spending on disease B. After disease B 
returned to baseline, the rational government response to 
the epidemic stopped. In the absence of the government 
response to epidemics of disease B, there is no natural way 


ADVANCING THE APPLICATION OF LMS THINKING IN HEALTH 
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Figure 6 Baseline scenario: results of NGO donation policies at various fixed levels of NGO donation per epidemic case of disease A 
from 0 to 30. Points are labeled with Da:Dg, which represent, respectively, the $ per additional DALY of disease A and of disease B. At all 
additional payments to the prevention workers, paying more reduces the burden of disease. 


the system would raise funding for PCWs to fight an epi- 
demic of disease B and the system would simply shift to a 
higher disease burden and not show return to equilibrium. 

Figures 6 and 7 plot societal dollars spent vs. DALYs 
averted. Note that the optimal zone in these figures 
would be at the bottom left where nothing is spent and 
there is no DALY burden. Points that are at the far 
right represent high disease burden and little spending. 
Starting from the bottom, as more is spent there is a 


rise in the vertical dimension and the DALY burden 
can become lower, moving to the left. As shown in 
Figures 6 and 7, there were diminishing returns to scale 
from NGOs investing in either disease — progressive 
amounts of money spent in the vertical dimension led to 
smaller and smaller reductions in DALY burden. Figure 6 
shows that, with spending on doctors held fixed, more 
spent on PCWs had a continuously increasing incremen- 
tal cost per DALY averted. 
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Figure 7 Baseline scenario: evidence of 
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Surprisingly, Figure 7 shows that spending on doctors 
generated negative returns with overall declines in popula- 
tion health per incremental $ spent on doctors’ services 
per epidemic DALY of disease A after spending surpassed 
a threshold. The harm is exclusively due to a higher DALY 
burden of unprevented cases of disease B. These cases 
of B were not prevented because the windfall payments to 
doctors during an epidemic of disease A were used to 
lobby the government for a larger share of the fixed 
health budget. An epidemic of disease A would trigger 
more fee based revenue for doctors, more NGO based 
revenue for doctors, and more government based 
revenue as a rational response. Whereas the NGOs and 
government were obligated to allocate funds rationally 
in proportion to the DALY burden, the sick patients, 
during epidemics of disease A, were not pursuing an 
optimal societal allocation of revenue to doctors. The 
patient revenue to doctors helped their lobbying power 
grow abnormally large. Patient-driven revenue led to so 
much doctor lobbying power that the prevention budget 
became squeezed towards zero and the burden of disease 
B rose. In models where Da was a lot higher than Dg, the 
iatrogenic epidemic of disease B could trigger remedial 
spending by the donor, but government health budgets 
had limited response to the epidemic due to doctor 
lobbying. Spending on doctors during the epidemic of 
disease A leads to doctors’ lobbying activity depleting 
the fixed government allocations to PCWs and leaving 
the population vulnerable to disease B. Without the 
agency of PCWs to prevent disease B, people perished 
in higher numbers. In Figure 7, the model shows nega- 
tive returns to investment in responding to cures with 
an inflection point occurring around $20 per DALY of 
disease A. Being willing to spend more than $20 per 
case of disease A in an epidemic of disease A creates a 
higher death rate from unprevented cases of disease B. 

Our sensitivity analysis, which focused on testing dif- 
ferent DALY weights for diseases A and B as well as 
disruptions in funding to either doctors or PCWs, con- 
firmed the robustness of the phenomenon of negative 
returns to spending high amounts per case of B. We 
observed the same findings despite 10% shifts in the 
DALY weight for either disease and despite 10% shifts 
in the government's standard response to epidemics, 
signaling that our conclusions were not simply a prod- 
uct of our initial set of parameters. 

Policy analysis explored what happens in a context 
where doctors lose their ability to earn consultation fees 
from seeing patients. By removing doctors’ disease- 
driven earning, and therefore making their income sym- 
metrical to PCWs, we saw the backward bending cost- 
DALYs effects observed in Figure 7 disappear. Another 
policy analysis removed all lobbying power of both doc- 
tors and PCWs. By removing lobbying from our model, 
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we no longer observed the backward bending effects seen 
in Figure 7. Either modification was enough to eliminate 
the damaging effects on population health of spending 
more money on curative services. Both solutions were 
effective and the results qualitatively resemble Figure 7 
(results available in a supplementary appendix). 


Discussion 

This system dynamics model of resource allocation ex- 
plicitly models political influences on the allocation of 
public spending in the health sector. It illustrates poten- 
tial unintended consequences from fully rational NGO 
contributions to respond to epidemics. Considerations 
from complex adaptive systems guided the exploration 
of non-linear, unintended consequences in a simulated 
health system — something that cannot be accomplished 
using traditional cost-effectiveness analysis techniques. 
The results offer several cautions about irrational distor- 
tions that occur in human systems even when rational 
policies are pursued. The donors in the simulation were 
efficiently allocating resources in proportion to the oppor- 
tunity to avert the most DALYs per dollar, but political 
lobbying resulted in their allocations to the doctors snow- 
balling into a large political lobbying fund that starved the 
preventive work of the PCWs and led to higher costs and 
unnecessary deaths. 

It is significant to note that the NGO's response to epi- 
demics of disease A did not factor in the fungibility of pri- 
vate spending on disease A. The NGO was programmed 
to assume that its donations were not being matched by 
private sector patient payments. If the NGO had been re- 
ducing its donations dollar for dollar in response to pri- 
vate spending, then the tragic loss of life from the crowd 
out of preventive spending would not have occurred. 
Traditional cost-effectiveness models have no way to 
simulate this type of unintended consequence because 
traditional cost-effectiveness measures costs, but says little 
about who will ultimately pay the cost. Furthermore, trad- 
itional cost-effectiveness cannot accommodate a positive 
feedback loop in which spending on a particular health re- 
source creates obligations to irrationally escalate spending 
on that resource due to political pressure from those who 
prosper from the initial allocation. 

Our findings add further insights into the debate on 
optimal resource allocation between curative and pre- 
ventive care. Because curative medicine offers services 
whose benefits are privately enjoyed by each sick person 
they are defined by economics as “excludable goods”. 
Market mechanisms work well to balance supply of re- 
source flows in proportion to the demand for excludable 
goods [28]. In contrast, the benefits of community 
prevention-work are non-excludable; everyone benefits 
but wants someone else to pay. With no defined prop- 
erty rights to transmit an effective demand signal to 
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suppliers, the system is prey to failure and mal- 
adaptation. The under-investment in community pre- 
vention joins a long list of many other market failures in 
health [28]. Community health advocacy falls to an as- 
sorted collection of community advocates, academics, 
and enlightened spokespersons [29]. In public policy- 
making it is common to hear advocates declare that 
their sector is “high priority” and requires special treat- 
ment [30]; it can be difficult at times to assess the legitim- 
acy of competing claims. However, the missing market for 
population-level prevention efforts and their history of 
extremely large benefits at low cost suggests that this 
might be an area that is perennially and inappropri- 
ately neglected. 

Our model’s focus on the asymmetric ability of the 
doctor class to achieve political clout from billable fees 
offers a cautionary lesson for NGO donors. When donors 
compassionately invest in curative strategies they may un- 
wittingly contribute to the rent-seeking capability of the 
doctors. Health systems can inappropriately de-prioritize 
prevention efforts delivered at population level. If it were 
easy to do so, an NGO or donor should stay informed 
about private revenue streams that could be augmenting 
their own allocation to address health problems. These 
private revenue flows can unbalance what could have been 
a measured response to a health issue, and require the 
donor to offset its contribution accordingly or focus to 
sub-populations that were not being covered privately. 

Funding for HIV/AIDS treatment programs in low- 
income countries may be an example where cure is 
crowding out prevention through a mechanism similar 
to the one in the model. Countries with limited human 
resources for health cannot absorb large amounts of 
donor spending on antiretroviral programs without staff 
redeploying away from their primary care jobs into the 
well-funded antiretroviral programs. Evidence of weaker 
performance on maternal and child health programs 
where HIV burdens are higher might be suggesting this 
sort of tradeoff [31,32]. 

Although we use the word “lobbying”, the basic dy- 
namics observed in the model do not require there to 
be an organized medical, pharmaceutical, or hospital 
association with a specific lobbying fund supporting 
politicians; real-world health care interest lobbies are 
much more subtle. Health care lobbying can be emotion- 
ally appealing and seductive. It is common for motivated 
disease advocates to spotlight a photo with a victim of 
a curable disease, or to note how a disease victim’s 
treatment is a human right [33,34]. Global donors are 
emphasizing disease-curing funds (e.g., providing AIDS, 
tuberculosis, and malaria treatments). The drive towards 
universal health insurance coverage is also an instance of 
lobbying that serves clinical interests while addressing 
rights to health care services. The growth of health 
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insurance can trigger a self-perpetuating feedback loop 
of political power by those who provide clinical care. 
One approach to protect prevention would be to ring 
fence prevention funds. Another approach would allo- 
cate donor funds preferentially to the neglected preven- 
tion sector as was done during a World Bank loan to 
Argentina [35]. Both of these approaches face challenges 
if the government is under high lobbying pressure. The 
ring fence can be punctured and the aid can be fungibly 
reallocated unless there is high commitment by the 
government to preserve a high priority sector [30]. 

Our model shows that the harmful spiral can be elimi- 
nated by eliminating patient fee payments or by eliminat- 
ing any lobby-type linkage between health worker revenue 
and government spending. Although these extreme sce- 
narios were modeled, total elimination is impractical in 
the real world. The reason to focus on patient fees is that 
it is only the fees that give doctors asymmetric growth in 
epidemic-driven revenue that is not enjoyed by PCWs. 
Epidemics of disease A enriched the doctors in payments 
from both doctors and the government. Epidemics of 
disease B enriched the PCWs only from government or 
donor funds because there could be no private market 
based on preventing disease B. The absence of a function- 
ing market for community prevention efforts requires a 
central planning function to step into the breach and dial 
up or down government spending based on epidemio- 
logical data and data on total health expenditure. Our 
paper emphasizes that this central intervention is prone to 
being captured and manipulated by selfish interest groups 
and requires institutional measures that insulate it. The 
National Health Service of the UK offers an example of a 
policy solution to this dilemma in which public health 
budgets are “ring-fenced” to prevent clinical enterprises 
from infringing on them [36]. 

Our model provides a quantitative application of complex 
adaptive system methodologies to health care systems and 
policy analysis [37-40]. It is an illustrative tool which was 
created to foster insight and understanding of unintended 
consequences from the financing of curative and preven- 
tion care. Preventive community-wide efforts in health 
labor under several disadvantages. When preventive ser- 
vices need to compete against cure for a common pool of 
government resources they are likely to suffer. They can 
be politically outgunned because curative health workers 
have the ability to amass more money through fees while 
community prevention workers cannot. A zero-sum game 
played between prevention and cure is not a fair contest. 
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Abstract 


Background: Health systems interventions, such as Capacity-building of health workers, are implemented across 
districts in order to improve performance of healthcare organisations. However, such interventions often work in 
some settings and not in others. Local health systems could be visualised as complex adaptive systems that 
respond variously to inputs of capacity building interventions, depending on their local conditions and several 
individual, institutional, and environmental factors. We aim at demonstrating how the realist evaluation approach 
advances complex systems thinking in healthcare evaluation by applying the approach to understand organisational 
change within local health systems in the Tumkur district of southern India. 


Methods: We collected data on several input, process, and outcome measures of performance of the talukas 
(administrative sub-units of the district) and explore the interplay between the individual, institutional, and 
contextual factors in contributing to the outcomes using qualitative data (interview transcripts and observation 
notes) and quantitative measures of commitment, self-efficacy, and supervision style. 


Results: The talukas of Tumkur district responded differently to the intervention. Their responses can be explained by 
the interactions between several individual, institutional, and environmental factors. In a taluka with committed staff and 
a positive intention to make changes, the intervention worked through aligning with existing opportunities from the 
decentralisation process to improve performance. However, commitment towards the organisation was neither crucial 
nor sufficient. Committed staff in two other talukas were unable to actualise their intentions to improve organisational 
performance. In yet another taluka, the leadership was able to compensate for the lack of commitment. 


Conclusions: Capacity building of local health systems could work through aligning or countering existing relationships 
between internal (individual and organisational) and external (policy and socio-political environment) attributes of the 
organisation. At the design and implementation stage, intervention planners need to identify opportunities for such 
triggering alignments. Local health systems may differ in their internal configuration and hence capacity building 
programmes need to accommodate possibilities for change through different pathways. By a process of formulating 
and testing hypotheses, making critical comparisons, discovering empirical patterns, and monitoring their scope and 
extent, a realist evaluation enables a comprehensive assessment of system-wide change in health systems. 


Keywords: Capacity building, District health system, Organisational commitment, Realist evaluation, Self-efficacy, 
Systems thinking, Programme theory 
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Introduction 

A capacity-building intervention that targets district 
health management teams is complex given that its im- 
plementation involves various actors with different ob- 
jectives, roles, and power. Further, the setting in which it 
intervenes is complex since district health systems are 
constantly evolving in response to national policies, the 
local socio-political environment, and internal dynamics 
within the healthcare institutions [1-3]. Realist evalu- 
ation can help to make sense of the complex nature of 
change that is expected in a scenario such as a district 
level capacity-building intervention. In this paper, we 
aim to demonstrate how the realist evaluation approach 
helps in advancing complex systems thinking in health- 
care evaluation. We do this by comparing the outcomes 
of cases which received a capacity-building intervention 
for health managers and explore how individual, institu- 
tional, and contextual factors interact and contribute to 
the observed outcomes. 


People at the core of health systems 

People are at the core of health systems capacity [4]. 
One of the characteristics of a well-performing health 
system is a robust human resources management system 
that ensures the right conditions to achieve and main- 
tain performance of the health workforce, which in- 
cludes health managers. Health worker performance is 
closely related to their management capacity, but not 
limited to capacity alone; performance of health staff is 
determined by a variety of factors related to motivation, 
organisational dynamics and culture, and environmental 
factors including socio-economic and political factors 
[5-7]. These determinants of performance are con- 
stantly changing. From a complex adaptive systems per- 
spective, capacity and performance could be viewed as 
emergent characteristics of a district health system that 
has many constantly self-adjusting and inter-dependent 
components [8]. 

From a realist perspective, it is not merely the imple- 
mentation of programmes, but people, who change 
things. A programme is expected to work through pro- 
viding new resources to one or more actors (agents) 
within this system. In response to the new resources in- 
troduced into the system by the programme, a change in 
the actors’ behaviour or their interactions with systemic 
elements could create a new way of doing things and thus 
result in the programme outcome. This “new way of 
doing things” is expected to result in better performance 
and hence better health services. While programmes 
could be designed to change behaviour of people through 
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introducing new knowledge, skills, or ideas, we see that 
in complex adaptive systems, the response of the people 
and the systems is neither straightforward nor easily 


predictable. 


Building capacity and improving performance 

Capacity building programmes are one of the most com- 
monly used strategies to improve performance of health 
workers, especially in low- and middle-income countries 
[1]. However, the connection between capacity building 
and performance is not straightforward; capacity build- 
ing is described as being multi-dimensional, spanning indi- 
vidual, teams, institutional, and health system dimensions. 
Experience from action research in several Indian settings 
has shown that the more we seek strengthening of systemic 
capacity, the more complex it seems to be and the harder 
it is to achieve, being rooted in organisational and the pre- 
vailing socio-cultural factors, while implementation of new 
skills and introduction of tools seem to be relatively less 
time-consuming and rooted in more technical domains 
[9]. In view of this multi-dimensional nature of health 
worker capacity (and performance), the implementation of 
capacity building interventions in district health systems 
is complex; improved performance may occur in some 
settings and not in others. Further, the transition from indi- 
vidual capacity to organisational capacity is not straightfor- 
ward; several organisational factors play a role in realising 
the individual capacity of health managers. The dispar- 
ity in results can be due to a variety of factors, including 
(but not limited to) the context and the actors’ percep- 
tions of the intervention and their responses to it, their 
interactions with each other, their organisation, and their 
environment. 


Complex adaptive systems: implications for programme 
evaluation 

The conceptualisation of district health systems as a 
complex adaptive system has implications for evaluating 
healthcare interventions. In this view, districts are sensi- 
tive to (dynamic) contextual factors as well as their ini- 
tial conditions, which accounts for the often differing 
outcomes of the same policy or programme. On the 
other hand, policies or programmes may produce similar 
outcomes through different organisational configurations 
within the same district [10]. The literature on programme 
evaluation as well as on complex adaptive systems urges 
evaluation researchers and practitioners to adopt research 
designs that allow the consideration of unanticipated ef- 
fects, adopting more flexible designs, capitalising on pat- 
terns and regularities emerging in the observations, and 
adopting an iterative manner of inquiry [2,11]. Studies 
that embrace complex adaptive systems thinking and 
theory-driven methods inherently allow for these as- 
pects as they invariably involve several cycles of 
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observations and analysis, especially in the complex 
healthcare settings. In public health, programme 
evaluation has embraced complexity. The recently re- 
vised Medical Research Council guidance for the as- 
sessment of complex interventions, for instance, calls 
for a closer examination of the causal mechanisms 
and theory-building to contribute to developing more 
effective interventions, and provide insight into how 
findings might be transferred across settings and pop- 
ulations [12,13]. However, flexible research designs for 
understanding change in response to interventions in 
a complex adaptive system may have trade-offs in 
terms of generating knowledge that has external valid- 
ity beyond the intervention being studied. In this 
paper, we present a case for using realist evaluation 
(explained below) to explain change within complex 
adaptive systems such as a district health system, 
while broadening the transferability of results [14). 


Realist evaluation and complexity 

The realist evaluation approach engages with complexity 
by taking an open systems approach to social systems 
[15]. The number of interacting agents, components, 
and forces that influence people and organisations in a 
given system is high, outcomes are sensitive to initial 
conditions, and thus outcomes are likely to show high 
variability. The realist approach to this complexity is to 
view reality as being stratified, with several layers of ex- 
planations to be found for the empirical observations. 
This provides a possibility to hypothesise and refine our 
explanations of why some phenomena occur [15,16]. 
In the realist view, there are many possible behav- 
ioural choices that people manifest (or not) in specific 
conditions, which results in the outcome. An evalu- 
ation using the realist approach thus begins by seek- 
ing an explanation for why the outcome of interest 
occurs in some places and not in others, keeping in 
mind that programmes work through people and their 
choices. Programmes facilitate agents to make choices 
and interact in new ways by providing physical or symbolic 
resources [17]. 

In order to understand the relationship between inter- 
vention, context, and outcome, realists use the concept 
of mechanisms, which are the “... underlying entities, 
processes, or |social] structures which operate in particu- 
lar contexts to generate outcomes of interest” 16]. In the 
case of complex adaptive systems, several latent mecha- 
nisms could be present within the system, which can be 
triggered by the intervention in the presence of specific 
contextual elements and result in the observed outcomes 
[18]. In practice, realists use the context-mechanism- 
outcome (CMO) relationship as a tool for empirical 
investigation and analysis. It allows for developing an ex- 
planatory theory of why the intervention worked for 
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some and did not for others (Figure 1). Theoretical ex- 
planations of this kind are referred to as middle-range 
theories, explanations which “...involve abstraction... but 
[are] close enough to observed data to be incorporated in 
propositions that permit empirical testing’ {16,19}. It 
should be noted that in the literature, middle range the- 
ory and programme theory are increasingly used inter- 
changeably. In this paper, for reasons of clarity, we will 
use the term programme theory. 

In a realist approach, the evaluation begins with for- 
mulating a programme theory (integrating the assump- 
tions of the programme designers and implementers 
with the existing wider knowledge or evidence on the 
topic and insight regarding the contextual factors that 
could affect the outcome). The programme theory is 
tested through empirical studies and a refined theory 
that explains why the intervention worked for some and 
not for others is the end point of the evaluation. This 
could be the starting point for a next study. Such cycles 
allow for fine-tuning of the programme theory and ul- 
timately to accumulation of insight. 

The seeking of an explanation for the patterns (or 
demi-regularities, which are somewhat predictable pat- 
terns or pathways of programme functioning) seen in 
some cases (and not in others) is the hallmark of a real- 
ist evaluation [14,21,22]. This addresses one of the fea- 
tures of complexity in social systems, wherein orderly 
patterns could be seen at the systems level, but often not 
at the individual level, due to reiterative positive and 
negative feedback loops among some components (and 
not in others) [23]. The foundations of realist evalu- 
ation within critical realism*, and its evolution as a 
scientific evaluation method are described by Pawson 
[14]. Its potential as an evaluation approach for com- 
plex health systems problems has gained interest over 
the last decade [24-28]. 

In this paper, we use a case study approach to explore 
how a capacity building intervention implemented in 
two different places in a district (both nested systems 
within the larger complex system of the district) evolved 
over time, using a realist evaluation, in order to under- 
stand how and why observed outcomes occurred. In line 
with the realist evaluation approach, cases were purpos- 
ively selected to allow testing of the programme theory 
propositions and to improve our understanding of why 
programmes work for some and not for others [15]. We 
then use the multipolar framework to summarise how 
the capacity-building intervention could have led to 
organisational change in a district health system. The 
multipolar framework, inspired by Champ et al. [29], is a 
heuristic tool that has been used to explain organisa- 
tional change in healthcare organisations in high-income 
settings with recent application in low- and middle- 
income country settings [22,30]. 
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Figure 1 The realist evaluation cycle showing the steps in a realist evaluation study. Figure based on steps described by Pawson and 


Study setting 
This study is based on a capacity building intervention 
in Tumkur district, which is one of the 30 districts in 
Karnataka state in southern India; Tumkur had a popu- 
lation of 2.67 million in 2011 [31]. It is an average dis- 
trict with respect to health and development indicators; 
it ranked 15th in the human development index ranking 
of the (then) 27 districts of Karnataka in 2005 [32 In 
Karnataka, poor health outcomes in maternal health 
have been attributed to systemic failures in managing 
health services and responding to critical problems ser- 
vice delivery [33]. Karnataka, like many other Indian 
states, lacks a management cadre within the health ser- 
vices. In Tumkur, as in all the other districts of the state, 
doctors with specialisation in one of the clinical special- 
ities and several decades of experience in hospital set- 
tings are appointed as health managers of districts and 
sub-districts without formal or in-service management 
training [34-37]. | 
The district health system in Karnataka is composed 
of several sub-systems called talukas. They are the polit- 
ical and administrative sub-units of the districts. In 
2011, the taluka population in Tumkur district ranged 
from 168,039 in Koratagere to 598,577 in the Tumkur 
taluka. Taluka health management teams are under the 
charge of a Taluka health officer (THO). An administra- 
tive medical officer (AMO) is in charge of the hospital, 
while the THO has the operational responsibility for the 
Primary Health Centres (PHC). The THO, AMO, and 
other members of the taluka health management team 
hold monthly review meetings of the taluka in which 


the block programme managers” and senior nursing staff 
participate. 
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A consortium of five non-governmental organisations 
partnered with the state government to organise a 
capacity building programme for health managers of 
Tumkur district. The programme consisted of periodic 
contact classes spread over 18 months (August 2009 to 
January 2011), periodic mentoring visits to participants’ 
workplace (till December 2011), and assignments to help 
participants apply the knowledge and skills discussed in 
the classroom teaching. The aim was to bring about or- 
ganisational change at the district level through improv- 
ing the performance of health managers with respect to 
planning and supervision of health services. The inter- 
vention identified capacitated health managers as the 
agency through which organisational improvement could 
be achieved. People were seen as being at the centre of 
organisational change. A much shorter intervention, con- 
sisting of a one-time five-day of contact classes for all the 
162 medical officers of the primary health centres of 
Tumkur district (all supervised by the health managers 
trained under the main intervention) and a facilitated 
discussion with Panchayati Raj Institution representa- 
tives (PRI), was also conducted. PRI representatives 
are members of the elected bodies of the local govern- 
ments at village and sub-district levels. The compo- 
nents of the intervention and the various actors involved 
are shown in Figure 2. A detailed description of the inter- 
vention and its implementation has been presented 
elsewhere [38,39]. 

In this paper, our purpose is to describe the complex- 
ity of a capacity-building intervention at the district level 
and illustrate the utility of the realist approach in ad- 
vancing the practice of systems thinking in complex 
settings. 
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Methods 

The realist cycle 

A realist evaluation begins with developing the initial 
theory. A programme theory is best considered as an ex- 
planatory pathway, connecting the inputs of the inter- 
vention to the expected outcomes, taking into account 
possible contextual factors and mechanisms [40]. The 
refining of the programme theory, starting from the 
initial programme logic of the designers, to a refined 
programme theory incorporating insight from literature, 
design of the programme, and its implementation con- 
text, is explained elsewhere [41]. Our refined programme 
theory was aimed at explaining the differences in taluka 
outputs following the intervention, accounting for differ- 
ences in the individual characteristics of the health man- 
agers, institutional factors within the two taluka health 
services and the differing environmental factors. The re- 
fined programme theory of the intervention that guided 
the choice of data and the analysis is shown in Figure 3. 


Case selection 

In the second step, cases were selected purposively. We 
assessed the performance of the 10 talukas of Tumkur 
district from 2009 to 2012, focusing on performance as- 
pects that could be logically connected to the capacity 
building intervention (using the programme theory of 
the intervention as a guide). We scanned taluka perform- 
ance with a focus on those showing least and most im- 
provement; we chose one positive and one negative outlier 
(contrasting case selection) for the analysis presented in 


Figure 2 Tumkur capacity building intervention: structure of the intervention, actors, and their roles. Government actors are shown in 
blue and non-governmental actors are shown in green. M stands for financial support, T for technical support and O for oversight. 


intervention referred to 
NRHM and later te KNSDRP 
fiate 2008) 


this paper. Figure 4 shows the talukas of Tumkur, includ- 
ing the taluka hospital and the PHCs. 


Data collection 

In realist evaluation, the choice of data to be collected is 
guided by the programme theory. First, we collected data 
on the intensity of the programme implementation: par- 
ticipation in classroom activities, frequency of mentoring 
visits, and retention of mentoring interest. The mentors 
seem to have preferred talukas based on their own as- 
sessment of interest shown for mentoring by the taluka 
team. Hence, retention of mentor interest has been chosen 
as a proxy for the taluka’s commitment towards the vision 
for change as articulated by the intervention. It was 
assessed on the basis of frequency of mentoring visits and 
observation notes of the mentors, and scored into high, 
moderate, and low. Second, we assessed intermediate 
outputs (self-efficacy, organisational commitment, style of 
supervision, and expression of intention of taluka man- 
agers to make changes) using data from a survey of health 
managers in Tumkur. 

Organisational commitment along with self-efficacy 
has been described as being crucial to performance and 
is considered as a key mechanism explaining human 
agency in various settings [42,43]. The three-component 
construct of organisational commitment by Meyer and 
Allen describes the nature of commitment of people to 
their organisations along three dimensions: affective com- 
mitment (emotional attachment to the organisation; a feel- 
ing of belongingness), normative commitment (a feeling of 
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Figure 3 The refined programme theory of the intervention showing possible intermediate steps between intervention inputs a 

expected outcomes. Data collected for the intermediate steps are shown. Grey boxes with stippled border show contextual cena identified 
as having an influence on the intervention outcomes during the refining of the programme theory. Unshaded boxes indicate the source of data. 
| Boxes shaded black indicate outcomes. Intermediate steps are shown in boxes shaded grey with no border. 


being obliged to the organisation), and continuance com- 
mitment (a feeling of being in the organisation because of 
a lack of alternatives) [44]; the three different dimensions 
of commitment co-occur. Self-efficacy was measured using 
a 10-item scale based on the Bandura scale [45] and degree 
of supportive nature of supervision was measured using a 
Likert scale questionnaire adapted from a tool by Oldham 
and Cummings and the Michigan Organizational Assess- 
ment Package [46,47]. The tools used have been described 
earlier and published elsewhere [39]. 

To assess the distal outputs of the intervention, we 
collected annualised data on budget utilisation, provision 
of 24/7 PHC services, coverage rates of institutional de- 
livery, delivery by caesarean section (CS), completion of 
three antenatal care visits, and immunisation. We also 
assessed changes in infant mortality rate and stillbirth 
rate from 2008 to 2012. Stillbirths and infant mortality 
reported in all the facilities of the taluka were used to 
calculate the rates. These quantitative data were supple- 
mented with qualitative data collected through. inter- 
views with health managers and observations. In-depth 
interviews were conducted with 21 health managers of 
Tumkur who participated in the intervention, their su- 
periors at state level (n=2), and their subordinates 
(PHC health staff and co-workers; n = 4). Participant ob- 
servation of monthly and annual review meetings at the 
taluka and district level was carried out to understand 
the organisational dynamics and the differences in inter- 
pretation and implementation of state policy. 


Analysis 
All interviews were transcribed and entered into NVivo 
10 (QSR International Ltd., Australia), together with the 


HEALTH RESEARCH POLIK Y AND SYSTEMS 


observation notes. During the analysis, we used the 
CMO as a heuristic tool (Table 1). These hypothetical 
CMO frames were based on the refined programme the- 
ory of the intervention, as described elsewhere [41]. Ini- 
tial codes reflected the programme theory elements of 
intervention, actors, context, mechanism, and outcomes, 
and new codes emerged. The quantitative data, including 
measurements of organisational commitment, self-efficacy, 
and style of supervision provided were integrated into the 
analysis and this helped in triangulating emerging findings. 
In this way, each case was analysed. 

We then compared the two talukas to further test 
whether the refined programme theory explained the 
differences in the outcomes. We supplemented these 
two contrasting case studies with demi-regularities from 
comparable settings in the other talukas. We focused on 
the internal dynamics within the taluka teams (micro- 
context) and the interaction of these teams with the 
immediate taluka environment (meso-context) and the 
larger policy environment at the district, state, and above 
(macro-context). We also described the organisational 
configurations of the two cases using the multipolar 
framework. 


Results 

Outcomes 

The responses of the talukas to the intervention varied, 
as shown in Table 2. The aggregated budget utilisation 
rate for Tumkur district® increased marginally, from 83% 
in 2009 to 85% in 2012. However, this conceals a variety 
of responses at taluka level. In Figure 5, the net annual 
change in utilisation (the net change in the proportion 
of available funds timely spent between two years) 
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Figure 4 Government health facility map of Tumkur showing the 10 ta/ukas, the hospitals (secondary care) and PHCs. Green ovals show 
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Figure 5 Annual change in utilization rate of selected talukas of Tumkur district from 2010 to 2012. The net change (from the previous 
year) in the aggregate budget utilization rates of all facilities in the talukas are shown for CN Halli, Tumkur, Sira, Gubbi, and Madhugiri ta/ukas. The 

District figures are for utilization rates of budget allocated for disease control programmes and other functions managed at the district level. 
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from 2010 to 2012, is shown. While some talukas, like 
Pavagada, improved their utilisation rate, others, like 
Madhugiri, reduced their spending rates. Yet others, 
like Turuvekere, showed wide changes from one year to 
another, while net change from 2012 to 2009 was only 
marginal. 


In Figure 6, the stillbirth rate in 2012 is plotted by 
taluka, against net change in stillbirth rate from 2009 to 
2012. We use the net change in stillbirth rates as a proxy 
indicator of performance. Stillbirth was chosen because 
of the emphasis in the intervention on using planning 
(through good annual situation analyses and problem 
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Figure 6 Stillbirth rates in 2012 by taluka shown against net change in this indicator from 2009 to 2012. Gubbi, Sira, Pavagada, and CN 


Halli stillbirth rates are labelied. 
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identification) and supportive supervision in improving 
maternal and child health outcomes. Such variability 
could result from several factors, including existing re- 
form processes that promote institutional deliveries, and 
improvements in the functioning of the health services 
(including the capacity building intervention). Besides 
such interventions, which influence all talukas to the 
same degree, context-specific socio-political factors and 
organisational factors, which are of interest in our evalu- 
ation lie, within the taluka health services and could in- 
fluence performance. We shall use the variability in the 
taluka level outcomes to purposively choose talukas and 
examine if the hypothesised explanations from the re- 
fined programme theory could explain these differences. 

In Table 2, the various individual, team, and institu- 
tional factors that we assessed based on the programme 
theory are shown. The factors chosen are a mix of indi- 
vidual and organisational contextual factors (intervention 
exposure, socio-economic development index of taluka, 
mentoring interest and supervision received, and team sta- 
bility), mechanisms of human agency at the individual level 
(intention to change, organisational commitment, and self- 
efficacy), and proxy measures of outcomes logically related 
to improvements in the talukas expected from the 
intervention as well as more distant taluka outcomes 
determined by several other factors. The talukas varied 
in their participation in classroom and mentoring ac- 
tivities, in view of transfer in and out of health man- 
agers in the taluka or absenteeism (either by choice or 
due to priority work at the taluka). Higher participa- 
tion in the intervention did not always result in an 
intention to make changes at the workplace (e.g., Gubbi 
and Tumkur with highest participation and only moderate 
expressions of intention for positive change); nor did ex- 
pressions of such intentions always result in improved out- 
comes (e.g., CN Halli with a 100% of the team expressing 
intent but showing negligible change over the three years 
in the outcomes). 

We purposively present the summary of the analysis of 
two contrasting cases — Gubbi and CN Halli — among 
the 10 talukas to illustrate how the CMO lens derived 
from our refined programme theory can be used to 
understand and explain how the outcomes in these cases 
could have come about and what could be the possible 
contribution of the intervention in these outcomes. We 
present the summary of the analysis of the empirical 
data in the form of observed outcome (O) in relation to 
mechanisms (M) and contextual conditions (C). 


Gubbi 

Gubbi’s stillbirth rate decreased the most among all 
the talukas in Tumkur; the improvements in proportion of 
CS performed and budget utilisation were modest (Table 2). 
Health managers from Gubbi participated actively in the 


32 HEALTH RESEARCH POLICY AND SYSTEMS 


Page 12 of 1 


intervention and retained the mentors’ interest. They 
showed relatively higher affective commitment than many 
other talukas (Figure 7). Only half of the health managers 
expressed an intention to make changes. | 

From the interviews and observations at Gubbi, the 
main theme emerging was commitment. The interest 
shown by the THO and the AMO towards improving 
services is evident from the interviews. The THO was 
given temporary charge of heading the team while sim- 
ultaneously being the medical officer of a nearby PHC. 
Yet, he felt that he could mobilise greater support to im- 
prove services in the taluka by motivating like-minded 
people. He felt that being a health manager is an oppor- 
tunity to bring about changes. 


“In my taluka for example, I think we can make big 
change. It is not that everybody in my taluka wants to 
make changes. Only one-third of them are motivated 
to make changes. And that is enough. I think I can 
make a lot of improvement by motivating these 
people.” 


~ Taluka health manager from Gubbi (g1) 


Such positive assessment of motivation of PHC staff as 
a strategy towards improving services was not shared 
widely in the other talukas. 

Both the AMO and the THO saw the intervention as 
an opportunity to benefit from the recent efforts to de- 
centralise the preparation of action plans to taluka and 
PHC level. They felt that the decentralisation of plan- 
ning under the National Rural Health Mission (NRHM)* 
was an opportunity to address specific problems at the 
PHCs. 


“More resources mean more opportunities to make 
change. If they slowly give more and more power to 
us at taluka level, we can make many more 
improvements. Right now, very little is possible at 
taluka level.” 


~ Jaluka health manager from Gubbi (g2) 


“NRHM has given block programme managers. This 
will improve plan preparation and monitoring. They 
are young and enthusiastic, but they need to some 
guidance and I think I can provide that.” 


~ Taluka health manager from Gubbi (g1) 


This general pattern of commitment at Gubbi is 
also seen in the Tumkur taluka, with a relatively high 
affective commitment, albeit with a higher turnover of 
staff. 


| 
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Figure 7 Boxplots of three dimensions of organisational commitment in the 10 talukas of Tumkur district. The three dimensions of 
commitment are based on Meyer and Allen [44]. AC is affective commitment, NC is normative commitment, and CC is continuance commitment. 
individual commitment measures for health managers were computed separately for AC, NC, and CC. For each taluka, box plots of the scores for 
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| each of these were plotted. 


The Gubbi pattern could be summarised as follows: in 
a decentralised taluka health system, committed health 
managers can make use of their increased management 
capacity to identify opportunities for improving their 
health services performance. 


Chikkanayakanahalli (CN Halli) 

While Gubbi is situated close to the district headquarter 
town of Tumkur, CN Halli is further away, but with a 
similar level of socio-economic development (Table 2). 
CN Halli showed hardly any change in most outcomes, 
in spite of a high intention among the health managers 
to make improvements in the taluka. CN Halli also had 
lower turnover rates of taluka level health managers. 
The level of affective commitment was comparable to 
that at Gubbi, but continuance commitment was rela- 
tively higher. 

CN Halli is amongst the most remote talukas. With a 
limited private sector, it is not a favoured choice of posting 
for doctors. For several months, the function of THO and 
AMO was taken up by the same person. The taluka level 
staff showed commitment towards the services and took 
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pride in working in a remote taluka with very limited 
human resources. However, during discussions about 
decentralised planning expressed by this taluka’s health 
managers, the dominant theme was frustration. 


“What PIP? What decentralisation? I sent so many 
requirements for staff and proposals for improvement. 
Only thing I got is more work, less staff and zero 
solutions. On one hand, I have to answer the local ZP’ 
members’ complaints and on the other hand, I have to 
just keep implementing plans and schemes coming 
from above. Nothing can be done without more staff.” 


~ Health manager from CN Halli (cnh1) 


While the decentralised planning brought about by 
NRHM was perceived as an opportunity in Gubbi, in CN 
Halli the respondents expressed frustration. This was also 
evident in several meetings at the taluka level, where a lack 
of power to make changes at the taluka and district level, 
for instance in recruitment of human resources and pur- 
chase of critical equipment, was often raised. 
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“NRHM has just brought more and more responsibilities, 
but no powers. For everything, we have to wait for a visit 
from the secretary or commissioner. More money means 

more work and more statements of expenditure and 


paperwork.” 


~ PHC health worker from CN Halli taluka at a 
review meeting (cnh2) 


Similar frustrations about increased paperwork and 
responsibilities were found in the thematic analysis of in- 
terviews and observation notes from Pavagada, another 
poorly staffed, and the most remote taluka in Tumkur. 


“The increased money with NRHM is good. But it’s not 
merely money. We need committed people who can 
stay in such a remote area. I am from this area and I 
live and work here. People who come here hardly stay 
beyond a few months. They either get frustrated or 
seek transfers.” 


—Health manager from Pavagada (P1) 


The recent reforms towards giving greater powers to 
the elected representatives were seen as a threat to their 
functioning. The taluka health staff felt that channelling 
the frustrations of the PHC staff upwards was their 
role much more than managing conflicts and frustra- 
tions or building amicable relationships with the elected 
representatives. 


“Nothing much can be done without giving powers at 
taluka level and PHCs. I cannot even appoint a Group 
D staff. Where is decentralisation in this?” 


~ a PHC staff from CN Halli taluka 


“What more can I do? I communicate promptly to my 
superior all the problems and I am still waiting for the 
solutions. In the [capacity building] programme they 
are saying, find local solutions. With so little staff, how 
much local solutions can I find? People just don’t want 
to work here. I handle two responsibilities at the same 
HUNG...” 


~ Health manager from CN Halli (cnh1) 


The pattern of CN Halli is also seen at the Pavagada 
taluka, which is also severely under-staffed, with a small 
group of health managers with comparatively lower 
levels of affective commitment. The improvements of 
the Pavagada taluka were poor, in contrast to the Sira 
taluka, which is also geographically remote and socio- 
economically poor, yet showing a remarkable vision in 
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the taluka team to operationalize emergency obstetric 
facilities in the hospital, a dire need in this remote re- 
gion. The Sira taluka, unlike Pavagada and CN Halli, 
was much more dominated by a continuance commit- 
ment rather than affective commitment. 


“We felt that we have to do it. So many mothers were 
just being referred to Tumkur. The delivery load is 
high and for several months, we had only one 
obstetrician, but somehow we managed. I know how 
the pressure is at the district hospital, so having LSCS 
facility at Sira decreases the burden at the district 
hospital. It’s not easy, but somehow it is happening.” 


~ Sira health manager (s1) 


The pattern of CN Halli could be summarised as 
follows: Health managers working in poorly resourced 
talukas, in spite of their improved management capaci- 
ties and intentions to make change, get frustrated by the 
lack of facilitating action from above. 


Discussion 

Health system interventions need to take into account 
the subunits of the local health system in which they 
intervene. In this case, each taluka can be conceived as 
a sub-system with a particular organisational context but 
a similar macro-context, exposed to the same inter- 
vention. In such cases, the realist evaluation approach 
helps to formulate specific CMO-based propositions 
that can be tested through comparing contrasting cases. 
This allows for building explanations on how organisa- 
tional change occurred in some settings and not in 
others. The process of testing and refining the CMOs al- 
lows for an understanding of the conditions through 
which such interventions could work in a complex local 
health system. 


Explaining change: contribution of the intervention 

While the training programme (the intervention) in- 
cluded all health managers in the district, their actual 
participation was variable. This depended on several fac- 
tors at the level of the participant (their interest and mo- 
tivation), distance between the taluka and the district 
headquarters, the staff turnover rate, and the responsive- 
ness of the implementers to the taluka teams. Many 
of these factors are related to each other, sometimes 
counterintuitively. For example, remote talukas like CN 
Halli and Pavagada had a relatively low turnover, while 
more sought-after talukas like Tiptur and Tumkur taluka 
had a higher turnover. Capacity building interventions that 
seek to strengthen local health systems ought to take into 
account such existing variations within the sub-systems at 
the design stage. 
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Health system strengthening interventions seek to 
strengthen core systemic functions of the local health 
system. The capacity building intervention sought to 
improve performance through improving planning and 
supervision. The contribution of such improvement (if 
any) ought to be framed against several other activities 
at the PHC, taluka, and district levels. For example, 
the provision of secondary level obstetric care at the 
taluka hospital includes developing the capacity of the 
facility to conduct CSs; this has been the policy focus 
in Karnataka for several years. In addition to the state 
government's pressure to implement this, health man- 
agers also face the pressure of the community and 
local elected representatives to operationalize CS facil- 
ities at taluka hospitals. However, in spite of favourable 
environmental conditions at the taluka level, effectively 
ensuring this requires a strong managerial vision and 
leadership; this was observed only in some talukas. This 
illustrates that, in a district health system influenced by 
several policies and environmental factors, it may be 
difficult to disentangle the contribution of the interven- 
tion to the observed outcomes. However, by choosing 
intermediate and distal outcomes at various levels (indi- 
vidual and institutional) that are most sensitive to the 
intervention inputs, it is possible to identify talukas 
where the intervention could have contributed to the 
outcome by seeking alignments with existing conditions 
and the characteristics of the people and teams in these 
talukas. 

Capacity-building interventions could work through 
identifying such existing alignments between local ac- 
tors’ needs, policy, and practice, and by strengthening 
conditions for the same. As the CN Halli case shows, 
in spite of favourable policy, community pressure, 
and a committed team at CN Halli, the frustrations 
of health managers resulting from previous negative 
experience with decentralised planning altered their 
choices and collective agenda-setting against actualis- 
ing CSs in their hospital. In contrast, health managers 
of Sira taluka showed relatively low levels of affective 
commitment and self-efficacy, but frustration was low. 
With the participation of elected representatives and 
through effective leadership by the AMO, the CS facil- 
ity was organised. Thus, in a taluka considered to be 
poorer than CN Halli in terms of socio-economic de- 
velopment indicators, the proportion of deliveries con- 
ducted by CS increased by 8.3% between 2009 and 
2012. Further thematic analysis of talukas that resem- 
ble some of the characteristics of our cases (such as the 
case of Pavagada discussed under the CN Halli case 
summary above) or are contrasting with our cases in 
some respects, could strengthen our findings and allow 
validation of these findings in future studies in similar 
settings. 
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From individual change to systemic change 

Although the capacity building intervention was imple- 
mented at the district level across all talukas, the expos- 
ure to the programme, the response to the intervention 
(attitudes towards change and intentions), the internal 
individual and organisational dynamics, and the out- 
comes, varied. These factors determine why programmes 
implemented at the district level may or may not achieve 
their expected outcomes, especially in those healthcare 
institutions where the conditions necessary for such a 
change do not exist. However, despite this potential for 
variation, formulating hypotheses in the form of CMO 
propositions and testing these empirically can help iden- 
tify patterns of response to intervention. The resulting 
CMO configurations can then be refined further by test- 
ing them in other cases of the district to arrive at an ex- 
planatory theory that elucidates what worked, for whom, 
and under what conditions. 

Capacity building interventions work through people 
and the choices they make. Many individual attributes, 
such as organisational commitment and self-efficacy, 
have been reported as mechanisms that explain human 
agency [42,44,52]. However, the taluka health system is 
more than a group of individuals with varying commit- 
ment or efficacy measures. The change in the organisa- 
tion comes about through the interaction among these 
participants, governed by rules and norms within their 
organisation (the organisational culture and their activ- 
ities that result in the organisational outputs), and the 
interaction between the organisation as a whole with the 
external environment. These relationships between the 
internal and external components of the organisation 
have been brought together in the multipolar framework 
for assessing performance of healthcare organisations, 
shown in Figure 8. The multipolar framework is based 
on Parsons’ theory of social action and inspired by the 
work of Champ et al. [29,30,53]. 

The observed changes in the talukas could be seen as 
having occurred through shifting or triggering of any 
of the six alignments in the multipolar framework. The 
taluka management team is responsible for managing 
not only the four core functions (the boxes in Figure 8), 
but also the alignments (the arrows in Figure 9) between 
the functions. The local configuration of these functions, 
and the management team’s response to tensions be- 
tween these functions explains the variation in the out- 
comes of the capacity building programme. 

The capacity building intervention sought to alter the 
outputs (service production) through increasing know- 
ledge and skills to develop annual action plans and 
supervision functions. An analysis of the programme 
theory of the intervention indicates that the designers of 
the intervention sought to bring about these changes 
through instilling a can-do attitude among the health 
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managers. This could be seen as trying to strengthen the 
allocation and operational alignments in the multipolar 
framework (Figure 8 and Figure 9). However, in the con- 
text of a health system that is undergoing decentralisa- 
tion to the district levels, and where participation of 
elected representatives within formal structures of the 
health service is being increasingly pushed for by the na- 
tional and state policy, the contextual alignment could 
dominate in some talukas, as was the case in CN Halli. 
However, a committed leadership at the taluka level 
could counter the negative perceptions of participation 
of elected representatives prevailing within the health 
service. In such cases (as in Gubbi; see Figure 9), the 
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legitimisation and strategic alignments could be trig- 
gered where the capacity building programme was seen 
as an opportunity to translate existing commitment to- 
wards the organisation into an improvement in its per- 
formance. The overall performance of the taluka is the 
result of how the alignments between the four poles are 
perceived locally and managed. The capacity building 
programme thus acts upon the taluka performance 
through imparting skills and vision to managers, who 
then balance or counter the emerging alignments. How- 
ever, it must be emphasised that in our study, the in- 
sights from the evaluation were not periodically fed back 
into the system to enable the local actors (implementers 
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Figure 9 The alignments that the intervention sou 
explain the responses of the cases are shown in red. 
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of the intervention and the recipient health managers) 
to benefit from or reflect on these. Realist evaluation 
could also be used as an entry-point for action research 
on local change, wherein the CMO frames being consid- 
ered or the refined programme theory could be shared 
periodically with local actors. Furthermore, such discus- 
sions and sharing with local actors could be further used 
to refine or validate the middle-range theory emerging 
from the evaluation. 


Realist evaluation and systems thinking 

Realist evaluation adopts a generative perspective on 
causality, according to which change occurs as a result 
of the interaction between actors within a specific con- 
text [54-56]. A programme theory that is constructed 
along these lines can be tested in a reiterative manner 
and allows for comparison across cases. The resulting 
insight, in the form of a refined programme theory, in- 
forms policymakers, managers, and funders on what 
works, for whom, in which conditions, and how. A real- 
ist evaluation of an intervention provides an explanatory 
theory on why the intervention worked for some and 
not for others through a process of adjudication between 
rival explanations. By employing the classical apparatus 
of the scientific method — “formulating hypotheses, mak- 
ing critical comparisons, discovering empirical patterns, 
and monitoring their scope and extent” — realist evalu- 
ation enables a comprehensive assessment of system- 
wide change [15]. 


Limitations 

The output of a realist evaluation is a programme theory 
or a middle-range theory (not a universal overarching 
theory), which provides a plausible explanation for the 
outcomes of the intervention; it cannot make predict- 
ive statements about the intervention. However, such 
middle-range theories form the basis for improving our 
understanding of complex interventions and help in im- 
proving design and implementation of such programmes 
in future. 

In this paper, outliers have been purposively selected 
based on outcomes that are logically connected to the 
intervention inputs. The explanation that we provide 
suffers from a possible confirmation bias. Ideally, a full 
realist evaluation needs to refine the middle-range the- 
ory through several iterations of cases selected based on 
diversity of outcomes. This will strengthen the explana- 
tory power of the middle-range theory. 

In an open systems world, there is no end to the ex- 
planatory possibilities and role of other mechanisms that 
can be put forth and tested. Hence, a major limitation of 
our evaluation is the number of such rival explanatory 
theories that can be practically put to test. While ac- 
knowledging this practical limitation, it may be said that 
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a critical mass of realist evaluations will strengthen the 
explanatory power of the middle-range theories tested 
by these evaluations [15]. 


Endnotes 

“Critical realism is a philosophical position in social 
sciences that approaches causation within the social 
realm as being possible through rationally choosing from 
rival theories, thus advancing the ‘explanatory power’ of 
theories. According to Pratschke (2003), in critical real- 
ism, “the ‘black-box’ of causation could be approached by 
understanding the gaps in the ‘generative mechanisms’ 
which may subsequently be explained by positing the 
existence of additional mechanisms at a deeper or more 
fundamental level” [57]. 

Block programme managers (BPM) are a new cadre 
of health managers created under the National Rural 
Health Mission (NRHM). These are young and typically 
recent graduates from management courses. BPMs oper- 
ate at the taluka level. Similar cadres of non-medical 
health managers were created at the district and state 
levels as well. 

“This was calculated by computing an average of per- 
centage utilization rates of budgets of all facilities in the 
taluka/ district. 

“The National Rural Health Mission (NRHM) is a flag- 
ship programme of the Indian government to strengthen 
government health services through greater financial al- 
location and human resources. Under the NRHM, there 
was an induction of new cadres of health workers and 
health managers from village level upwards to PHC, 
taluka, district, and state levels. Decentralised planning 
and increased participation of elected representatives in 
formal structures within health services were key fea- 
tures of NRHM. 

“PIP stands for programme implementation plan. The 
PIP is the annual action plan instituted by the NRHM. 
As per the NRHM, the PIP is an instrument for decen- 
tralised planning. 

‘ZP stands for Zilla Panchayat, the local governments 
at the district level. 
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Abstract 


Background: Although there is widespread agreement that strong district manager decision-making improves health 
systems, understanding about how the design and implementation of capacity-strengthening interventions work is 
limited. The Ghana Health Service has adopted the Leadership Development Programme (LDP) as one intervention to 
support the development of management and leadership within district teams. This paper seeks to address how and 
why the LDP ‘works’ when it is introduced into a district health system in Ghana, and whether or not it supports systems 
thinking in district teams. 


Methods: We undertook a realist evaluation to investigate the outcomes, contexts, and mechanisms of the intervention. 
Building on two working hypotheses developed from our earlier work, we developed an explanatory case study of one 
rural district in the Greater Accra Region of Ghana. Data collection included participant observation, document review, 
and semi-structured interviews with district managers prior to, during, and after the intervention. Working backwards from 
an in-depth analysis of the context and observed short- and medium-term outcomes, we drew a causal loop diagram to 
explain interactions between contexts, outcomes, and mechanisms. 


Results: The LDP was a valuable experience for district managers and teams were able to attain short-term outcomes 
because the novel approach supported teamwork, initiative-building, and improved prioritisation. However, the LDP was 
not institutionalised in district teams and did not lead to increased systems thinking. This was related to the context of 


high uncertainty within the district, and hierarchical authority of the system, which triggered the LDP’s underlying goal of 
organisational control. 


Conclusions: Consideration of organisational context is important when trying to sustain complex interventions, as it 
seems to influence the gap between short- and medium-term outcomes. More explicit focus on systems thinking 


principles that enable district managers to better cope with their contexts may strengthen the institutionalisation of the 
LDP in the future. 
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; Correspondence: aku.kwamie@wur.nl 

suhiversity of Ghana, School of Public Health, P.O. Box LG 13, Accra, Ghana 
Wageningen University, Anthropology and Sociology of Development, P.O. 
Box 8130, Hollandseweg 1 6700, EW, The Netherlands 


) & Caneel 2014 . licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative 
CooVed ommons Attribution License (httn//creativecc Ons lic py/2.0). whic istribut 
7 mons Attribution License (nttp//creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and 


2 seh in a medium, provided the original work is properly credited. The Creative Commons Public Domain 

| >, om os ~~] x . fo + - y . : 
Dedication waiver (http//creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article 
unless otherwise stated ) | 


40 HEALTH RESEARCH POLICY AND SysTEMs 


Kwamie et al. Health Research Policy and Systems 2014, 12:29 
httpy/www.health-policy-systems.com/content/12/1 /29 


Background 


To date, the majority of management and leadership initia- 
tives in low- and middle-income countries (LMICs) have 
focused on skills acquisition [1], with less attention paid to 
the complexity of the contexts and the health system ar- 
rangements which support or hinder such initiatives, In this 
paper we explore, using realist evaluation methodology, the 
outcomes, contexts, and mechanisms of a management 
and leadership initiative introduced into the district health 
system in the Greater Accra Region of Ghana, and whether 
or not such an intervention supports systems thinking in 
district managers. Firstly, we present an extensive back- 
ground of the decision-making context at district-level. We 
then delve into several concepts, including systems thinking 
and continuous quality improvements, before discussing 
our analytical framework, case study design, results, and 
conclusions. 


District manager decision-making and systems thinking 
In district health systems in LMICs, district managers 
link the national and regional levels — where policies are 
formulated — to the facility and community levels — 
where services are delivered. District managers are re- 
sponsible for providing management and leadership to 
supervise staff, balance resources, coordinate pro- 
grammes, and network with local officials and commu- 
nity members, all in a specific time and place. Thus, the 
manner in which district managers make decisions is 
important. It has been argued that limited management 
and leadership capacities at district level contribute to 
bottlenecks in achieving health outcomes [2-5]. 
Questions pertaining to management and leadership are 
some of the most complex in health systems analyses, not 
least because developing management and _ leadership 
requires nurturing myriad individual and organisational 
capacities [6]. Consequently, interventions that aim to 
strengthen management and leadership are also complex, 
and engage with both individual and _ organisational 
processes. District managers find themselves navigating 


Table 1 Systems thinking skills 


From ‘usual thinking’ approaches... 
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complex environments in which district health systems dis- 
play features of complex adaptive systems, such as self- 
organisation, path-dependence, emergence, and feedback 
loops. District health systems evolve over time as a result of 
multiple interactions between individuals and the system's 
structure [7,8]. As an approach to navigating this complex- 
ity, systems thinking aims to identify the interrelations be- 
tween a system's various components [9]. Defined by de 
Savigny and Adam, “systems thinking is an approach to 
problem-solving that views ‘problems’ as part of a wider, dy- 
namic system” |10]. The authors further identify a cluster of 
problem-solving skills relevant for systems thinking that 
distinguishes it from ‘usual thinking’ paradigms (Table 1). 
Due to their vantage point at the helm of district health sys- 
tems, systems thinking can usefully support district man- 
ager decision-making. 


Continuous quality improvements 

Continuous quality improvement (CQI) is both a man- 
agement philosophy and approach. Adopted in Ameri- 
can healthcare institutions in the 1980s, the concept 
spread to LMICs during the 1990s. CQI offers a system- 
atic way of supporting change in management processes 
towards improving the organisational culture of quality 
[12]. CQI is based on the assumption that problems 
within organisations are not rooted clinically or adminis- 
tratively, but are rather systemic and arise out of struc- 
tural inabilities to perform as intended [13]. McLaughlin 
and Kaluzny identify nine elements necessary to classify 
an approach as CQI (Table 2). Systems thinking is em- 
bedded within this constellation, and can be seen as the 
glue that binds CQI elements together (the authors refer 
to this as ‘systems-view’). 

To date, the impact of CQI in sub-Saharan Africa has 
been mixed. Case studies from three countries have dem- 
onstrated several factors that contribute to reduced CQI 
sustainability and effectiveness [14]. These are: i) introdu- 
cing quality management as a vertical programme; ii) 
lacking systemic perspectives and identifying problems in 


...to systems thinking 


Focused on particular events (Static thinking) 


Problems framed in terms of a patterns of behaviour over time (Dynamic 


thinking) 


Focused on particular details (Tree-by-tree thinking) 


Focused on factors that influence/correlate with results (Factors 
thinking) 


Systern-generated behaviours are driven by external forces (Systems- 
as-effect thinking) 


Causality is viewed as uni-directional, without interdependence or in- 
teractions between causes (Straight-line thinking) 


(Adapted from Richmond, 2000 [1 1)). 


Focused on understanding the context of relationships (Forest thinking) 


Focused on causality and understanding how behaviour is generated 
(Operational thinking) 


System-generated behaviours are driven by internal actors who interact with 
system itself (Systems-as-cause thinking) 


Causality is viewed as ongoing with feedback effects, including 
interdependence and interactions between causes (Loop thinking) 
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Table 2 Elements of continuous quality improvements — 
Description —  —i 
Emphasis on analysis of the whole system 
providing a service, or influencing an outcome 


Element 


Systems-view 


Emphasis on both customer (patient, provider, 
payer) satisfaction and health outcomes as 
performance measures 


Customer focus 


Emphasis on gathering and use of objective data 


Data-driven 
analysis on system operations and system performance 

_ Implementer Emphasis on involving the owners of all 
involvement components of the system in seeking a common 


42 


understanding of its delivery process 


Emphasis on identifying the multiple root causes of 


Multiple causation 
a set of system phenomena 


Solution Emphasis on seeking a set of solutions that 

identification enhance overall system performance though 
simultaneous improvements in a number of 
normally independent functions 

Process Emphasis on optimising a delivery process to meet 

optimisation customer needs regardless of existing precedents, 
and on implementing the system changes 
regardless of existing territories and fiefdoms 

Continuing Emphasis on continuing the systems analysis, even 

improvement when a satisfactory solution to the presenting 


problem is obtained 


Emphasis on organisational learning so that the 
capacity of the organisation to generate process 
improvement and foster personal growth is 
enhanced 


Organisational 
learning 


(Adapted from McLaughlin and Kaluzny, 1994 [13]). 


their own sub-systems; iii) oversimplifying decision-making 
through the use of toolbox techniques; and iv) the conun- 
drum of organisational culture and quality management: 
does organisational culture change to modify practise, or 
does organisational culture change by modifying practise? 
Furthermore, CQI is always implemented within an organi- 
sation’s own context — its history, cultural norms, and 
values. This latter point contributes to understanding the 
‘inherent duality’ of CQI, namely that its principles are 
based on two distinct, paradoxical goals: although CQI pro- 
motes organisational control, uniformity, and standardisa- 
tion, it also gives rise to organisational creativity, learning, 
and cultural change. This means that CQI practise (and the 
mechanisms behind it) will vary depending on whether its 
underlying goal is organisational control or organisational 
learning. Related to this, CQI’s underlying goal will be 
driven, either implicitly or explicitly, by the culture and 
structure of the organisation itself. Sitkin et al. [15], suggest 
that the most likely goal is informed by the degree of organ- 
isational uncertainty: when uncertainty is high, the organ- 
isation is predisposed to learning because control, in a 
sense, is out of reach. On the other hand, when contextual 
uncertainty is low, the organisation is predisposed to con- 
trol because the problem is well understood and can be 
dealt with mechanistically. CQI has been Proposed as a po- 
tential solution to improving service delivery in Ghana [16]. 
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Implementing the Leadership Development Programme 
in the Greater Accra Region 

In Ghana, district managers are staff of the Ghana Health 
Service (GHS)*. The context of district manager decision- 
making is such that resource decisions (human, material, 
and financial) are constrained. This is partly due to the hier- 
archical structure of the GHS in which decision-making re- 
mains highly centralised, and resources scarce [17,18]. 
District managers have more discretion around program- 
ming decisions. Formalised management training is limited, 
and most managers learn their management roles on 
the job. Additionally, managers face serious time con- 
straints due to concurrent scheduling of vertical and donor 
programme activities. 

The LDP has been intermittently implemented in Ghana 
since 2008. Developed by Management Sciences for Health 
[19], the LDP has been implemented in several countries 
including Egypt [20], Kenya [21,22], and Mozambique [23]; 
the Greater Accra Region first introduced the LDP in 2010. 
In 2011, the LDP was proposed as an approach to address 
limited responsiveness, lacking leadership, and mismatched 
resources indicated as bottlenecks to improving maternal 
and newborn (MNH) service delivery [24]. The LDP is de- 
signed for teams to apply ‘leading and managing’ practices 
to service delivery problems (referred to as ‘challenges’ in 
the LDP — Table 3). This is realised through teamwork, de- 
fining root causes, action planning, monitoring, and evalu- 
ation, and repeating the cycle. Its programme theory puts 
forth that, when deployed in tandem, leading and managing 
practices improve work climate, management systems, and 
capacity to respond to change, and ultimately result in bet- 
ter services and health outcomes. However, the programme 
theory is based on LDP content alone and does not account 
for differential impacts in various contexts, 

A review of the LDP suggests that it draws upon CQI 
principles in its approach. The LDP acknowledges the com- 
plex environment of managerial decision-making, and states 
that sustaining advances in health outcomes only occurs 
when leading and managing practices are absorbed into rou- 
tine practise (ie., their institutionalisation). However, the 
LDP is not explicit about this theoretical basis in CQI, nor 
does it claim systems thinking as a prime objective. We rec- 
ognise that CQI philosophy — and implicitly, systems think- 
ing — is embedded within the LDP practise and tools, and 
we were therefore interested in understanding the degree to 
which the LDP can stimulate systems thinking in district 
teams. Though the language differs, the concepts of systems 
thinking, CQI, and the LDP overlap in their approach to 
shifting problem-solving towards a more systemic orienta- 
tion for improved decision-making: if systems thinking is 
the Capacity to see interrelationships between components 
of a system, CQI is the process of managing these interrela- 
tionships, and the LDP is a practical intervention to imple- 
ment these principles. This overlap is illustrated in Figure 1. 
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Table 3 LDP leading and managing practices 


identifying client priorities and needs 
Seeing opportunities, trends, constraints and risks 


(Organisational outcome: valid, current knowledge of 
context) 


Focusing Developing shared goals 


(Organisational outcome: articulated Mission, vision, 
strategies and priorities) 
Aligning/ 


Building congruence between values, mission, 
mobilising 


structures and daily actions 
Supporting teamwork 


(Organisational outcome: external and internal 
stakeholders have ownership over organisational 
goals and support resource mobilisation towards 
these goals) 
Inspiring Building trust and acknowledging team members 
Modelling creativity and learning 


(Organisational outcome: climate of continuous 
learning with committed staff) 


Managing practices 


Planning Identifying goals, annual plans and performance 


objectives 


(Organisational outcome: defined results and 
matching resources) 
Organising Ensuring accountability and authority structures 
Aligning staff capacities with planned activities 


(Organisational outcome: functional structures and 
processes for operations) 
Implementing Integrating workflows and systems 
Balancing competing demands 


(Organisational outcome: effective, efficient and 
responsive actions) 


Monitoring and Reflecting on progress against action plans 


evaluation 
Improving work processes and procedures 


(Organisational outcome: continuous up-to-date data 
for decision-making) 


(Adapted from Mansour et al., 2005 [19]). 


The objectives of our study, therefore, are to understand: 
a) the mechanisms by which a complex intervention intro- 
duced into a complex context brings about its observed 
outcomes (i.e., how and why does the LDP ‘work’ when it is 
introduced into a district health system in Ghana); and b) 
whether or not the LDP increases systems thinking in dis- 
trict managers in this context. 


Methods 

Study setting 

The LDP was introduced in a rural district, Dangme West". 
The study took place from November 2011 to August 
2013. An initial period of participant observation prior to 
the onset of the intervention lasted from November 2011 
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to January 2012. The LDP intervention was implemented 
from February to August 2012, and a follow-up period of 
participant observation extended until August 2013. 
District managers were defined as: i) members of the 
district health management team (DHMT); ii) members 
of the district hospital management team; and iii) mem- 
bers of the three sub-district health teams. These man- 
agers were selected because they represent top-level 
management for decision-making within the district. 


Description of the LDP intervention in Dangme West 

The LDP was introduced to district teams (district 
health administration, district hospital, and three sub- 
district management teams) by a facilitation team con- 
sisting of three members of the regional health adminis- 
tration, and one external consultant specialised in the 
LDP. The curriculum, teaching materials, and learning 
strategies were based on the LDP Handbook [19]. The 
LDP consists of a six-month cycle of root challenge 
identification, action planning, and monitoring and 
evaluation. For each training workshop, district teams 
consisted of 4 to 7 members per team, depending on the 
size of the facility. These were managers (core manage- 
ment including medical superintendents, district direc- 
tors of health services, deputy directors of nursing 
services, physician assistants, and hospital administra- 
tors), and staff (accountants, public nursing officers, and 
midwives). Two-day, face-to-face workshops were held 
in the capital city Accra three times bi-monthly. These 
involved modules on LDP practices, developing a shared 
team vision, diagnosing challenge root causes, develop- 
ing action plans, setting priorities, mobilising stake- 
holders to commit resources, monitoring and evaluation, 
understanding roles in teamwork, and building trust. 
Workshops were interspersed with monthly coaching 
visits, with the facilitation team attending teams and 
their wider staff in their facilities to ensure organisation- 
wide diffusion of LDP teachings. For their LDP results, 
each team identified one MNH-related challenge they 
wished to address (Table 4). Every team attained their 
planned results except for one sub-district team; at the 
time, the health facility did not have a resident midwife 
and thus faced difficulties in improving its skilled deliv- 
ery coverage. 


Study design: realist evaluation 

We used a case study design as most appropriate for organ- 
isational studies in which ‘how or ‘why’ questions are being 
asked. Criticisms of case study designs include their weak 
external validity [25]. Seeking to address this criticism 
through cumulative validation, realist evaluation is an ap- 
proach capable of addressing complex investigation and 
probing causal linkages between contexts, actors, and the 
changes observed. Realist evaluation attempts to move 
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beyond asking ‘did the intervention work? towards under- 
standing ‘how did the intervention work, for whom, and in 
which contexts’ [26-29]? The case study begins with the 
formulation of the middle range theory (MRT), based on 
existing theory and past actor experience. The MRT, struc- 
tured as a ‘context + mechanism — outcome’ (CMO) con- 
figuration, is validated with actors, and against the 
literature. The validated MRT then becomes the working 
hypothesis to be ‘tested’ in the case. It is subject to revision 
based on accumulated new evidence. 


Analytical framework: our middle range theory 
Context of district manager decision-making (C) 
The first part of developing our MRT included an in- 
depth exploration of the decision-making context for 
district managers in Ghana. Based on our pre-LDP ob- 
servation period, we found that district managers have 


narrow decision-space due to the highly-centralised au- 
thority within the GHS. National-level control over re- 
sources leads to resource uncertainty at district level. 
Through formal and informal communication channels, 
district managers engender trust and employ it as a cop- 
ing mechanism to counter organisational uncertainty 
and manage the risk of not fulfilling their managerial 
mandates of oversight, coordination, and networking in 
the face of resource scarcity. Trust and respect for 
regional- and national-level authorities further legiti- 
mises the system’s hierarchy, thereby reinforcing it [un- 
published observations]. This decision-making ‘loop’ is 
the context into which the LDP was introduced. 


Outcomes of the LDP - short- and medium-term (O) 
Furthermore, we worked backwards from the observed 
short-term outcomes of the LDP (ie, LDP results) and 


Table 4 LDP results (short-term outcomes) February to August 2012 


Team 


LDP challenge 


LDP results (short-term outcome) 


District Health Administration 
District Hospital 

Sub-district 1 

Sub-district 2 


Sub-district 3 Increase focused a 
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Increase skilled delivery from 37% to 40% 
Reduce still birth from (n=) 30 to 20 
Increase skilled delivery from 15% to 18% 


Increase skilled delivery from 1.7% to 5% 


Increased skilled delivery to 51% 
Reduced still birth to (n=) 11 

Increased skilled delivery to 19% 
Increased skilled delivery to 2.6% 


Increased focused antenatal care to 22% 
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medium-term outcomes, which were interpreted as the re- 
sidual organisational changes (i.e., LDP institutionalisation), 
These included new organisational roles and relationships 
as a result of the LDP, extensiveness (ie., how widely dis- 
seminated across the organisation) and intensiveness (ie. 
how deeply integrated into routine practise) of the LDP, 
and any organisational routines displaced by the LDP [30]. 


Mechanisms of the LDP (M) 

Through our MRT, we attempt to uncover the mecha- 
nisms of the LDP. Our beginning assumption was that if 
systems thinking took place as a result of LDP practices, 
this would support LDP institutionalisation. In a feed- 
back mechanism, institutionalisation of the LDP would 
further increase systems thinking. We hypothesised our 
MRT as follows: 

The LDP brings about its short-term outcomes by en- 
couraging district managers to seek alternative sources 
of financial and material resources. If successful, the in- 
creased ability to look within and across the district for 
resources: i) supports relationship building with district 
stakeholders, which improves the number and quality of 
district relationships; ii) expands managerial understand- 
ing of the linkages and interactions in the district health 
system, which deepens systems thinking in managers, 
and supports LDP institutionalisation; and iii) reduces 
resource uncertainty, which lessens managerial risk, and 
thus the need to draw upon trust and respect as coping 
mechanisms. Reduced resource uncertainty increases 
district manager decision-space. Reduced uncertainty 
triggers the LDP’s underlying focus on organisational 
control. 


Rival MRT 

We also propose a rival MRT where the LDP brings 
about its short-term outcomes by reinforcing hierarch- 
ical authority, because it is introduced in a top-down 
manner. As such, resource uncertainty remains high, 
and district manager decision-space narrow. Thus, dis- 
trict managers continue to rely on trust and respect as 
coping mechanisms to deal with resource uncertainty. 
The context of high uncertainty triggers the LDP’s 
underlying focus on organisational creativity. This focus 
on creativity stimulates systems thinking, which supports 
LDP institutionalisation. 


Data collection 

Document review 

For data on the LDP implementation, we reviewed weekly 
district management team meeting minutes and monthly 
regional management team meetings for the duration of 
the study period, as well as all training workshop materials, 
team presentations and action plans, and reports from pre- 
vious LDP cycles in other regions. For overall context, we 
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further reviewed national, regional and district policies, and 
protocols (Additional file 1: Table $1). 


Participant observation 

For the duration of the study period, the first author partici- 
pated in weekly district health management meetings, 
monthly regional health management team meetings, semi- 
annual district planning and district review meetings, all 
LDP training workshops and coaching visits, teams’ LDP 
activities, DHMT supervisory visits to sub-districts, and 
day-to-day operations of the district. Until October 2012, 
the third author participated in monthly regional health 
management team meetings. Continuous discussion with 
management and staff was the method of sense-making 
used. As part of their routine management meetings, valid- 
ation workshops took place at the end of the initial and 
follow-up observation periods to feedback findings to dis- 
trict teams and integrate their views into the analysis. 


Semi-structured interviews 

We conducted a total of 23 interviews with members of the 
DHMT (8), district hospital management (4), and sub- 
district management (7); 4 managers were lost to staff 
transfers (2 from the DHMT and 2 at the sub-district level). 
At the regional level, we interviewed 3 out of 4 members of 
the LDP facilitation team, and one development partner 
supporting the LDP; 17 respondents were women and 6 
were men; 3 respondents were in their current posting less 
than 1 year, 13 between 1-3 years, and 7 between 3-5 
years. More than half the respondents (12) had no prior 
formalised management training. 

Interview guides were developed to investigate team 
perceptions of quality, actual LDP implementation (in- 
cluding challenges and functioning), influence of concur- 
rent district initiatives, organisational sustainment of 
LDP practices, and changes in relationships and re- 
sources. Interviews took place 8 months after the end of 
the LDP. 


Data analysis 

Audio-recorded interviews were conducted in English, 
and observational field notes were converted into tran- 
scripts, cleaned, and entered into Atlas.ti® qualitative 
analysis software. Transcripts were coded against an ini- 
tial start-code list developed from systems thinking, 
LDP, CQI concepts, and our MRTs. Emerging themes 
from the data were also coded. In order to ‘configure’ 
our CMOs [31], we began with the short-term out- 
comes. We triangulated across data type and source to 
systematically arrange our medium-term outcomes and 
unearth potential mechanisms of the LDP. We then 
drew out linkages between the contexts, outcomes, and 
identified mechanisms in a causal loop diagram (CLD). 
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Ethical considerations ake 
This study was part of a larger study to identify effective 
ways of improving MNH service delivery, for which eth- 
ical approval was awarded by the Ghana Health Service 
Ethical Review Committee. Teams were made aware of 
the observation periods. Respondents participated volun- 
tarily, and were able to withdraw at any time. Informed 
consent was obtained from all respondents, and re- 
spondent anonymity was maintained during all parts of 


the study using coding. 


Results 

LDP as it was implemented 

The LDP was mainly implemented as designed. During 
implementation, the LDP was frequently discussed as 
part of management team meetings, and was often men- 
tioned at the monthly regional health management team 
meetings. 

The LDP training approach was more team-based, less 
didactic, and more intensive than most district work- 
shops. Modules focused more on the deployment of 
LDP tools and proceeding through LDP processes, and 
less on facilitating teams to reflect on their own organ- 
isational practices or thinking systemically through 
them. This was indicated in the first LDP workshop, 
where facilitators identified the programme goals as be- 
ing: i) to learn how to lead and manage to enable others 
to face challenges and achieve results; ii) to apply tools 
to analyse challenges to achieve results; iii) to know how 
to produce measurable results; and iv) as managers, to 
learn how to build a positive work climate. The em- 
phasis was more on the LDP’s ‘managing’ rather than 
‘leading’ practices. 

A review of teams’ LDP action plans and presentations 
showed that teams broadly undertook two categories of 
activities: i) community sensitisation and customer care 
training for frontline staff, or ii) lobbying local organisations 
for material resources. From the customer care training 
workshops we found clear patterns of hierarchy being 
reinforced. Customer care workshops were facilitated by 
non-LDP regional staff and were regarded as ‘customer care 
as corporate responsibility, to redeem the corporate image’. 
In part, this stemmed from some high-profile media cases 
about staff error. Emphasis was placed on rules and regula- 
tions of the GHS, proper comportment of staff in forms of 
address towards their seniors, and dress codes. Very little 
related to client-provider relationships and there was min- 
imal opportunity for staff to reflect on their experiences 
with clients. Furthermore, in performing their root cause 
analyses, teams were not trained to investigate the interrela- 
tionships between different causes, but rather to deal with 
single root causes separately.’ Taking the example of poor 
staff attitude, teams worked through their root cause ana- 
lysis in the following manner: 
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“staff attitude is poor, because staff lack courtesy and 
good customer care; this is because they have 
inadequate knowledge about good customer care; 
which is because they have not been trained on good 
customer care; therefore the solution is to provide 
customer care training.” 


An example of gaps between LDP practise in the con- 
text of its implementation and LDP practise in routine 
work was observed 2 weeks prior to the final LDP work- 
shop. In one sub-district, having been called to assist in 
a conflict between staff and management, DHMT mem- 
bers resolved the situation by stating: 


“Any time your leader tells you something, she has a 
plan. Only one person can lead, others follow 
faithfully. Yours is to do what you are told. The rest, 
she will manage’. 


Once ended, there was little evidence of teams’ efforts to 
support LDP institutionalisation. None of the five teams en- 
gaged in another LDP cycle, no new staff were oriented in 
the LDP, no funds were set aside for LDP activities, and 
meeting minutes and staff conversations no longer reflected 
mention of the LDP. The lack of team efforts towards LDP 
institutionalisation was influenced to some extent by time 
constraints of routine district work: at the time the LDP 
ended (August-September), district teams were focused on 
completing year-end activities and reporting, and preparing 
for a new planning cycle. LDP institutionalisation was fur- 
ther compromised by changes of leadership at regional, 
district, and sub-district levels, which witnessed the ap- 
pointment of new directors at each level. Critically, the 
splitting of the district into two separate districts in October 
2012 required new administrative structures in the new dis- 
trict, and a restructuring of relationships across both dis- 
tricts. It does not appear that teams used their LDP 
practices to support these transitions. Several months after 
the end of the LDP, the majority of team members could 
not list the LDP practices. The LDP did not appear to 
support the development of systems thinking in district 
managers. 


Participant perceptions of the LDP 

The introduction of the LDP from the region was unex- 
pected by district teams, and was not initially part of their 
annual work plan. However, in the context of verticalised 
programming, this is common. The facilitation of the LDP 
by the region was perceived in two distinct ways. From the 
regional perspective, facilitating the LDP provided an op- 
portunity to remind district teams of ‘proper conduct; part 
of which was complying with regional directives. From the 
district perspective, having regional facilitators participate 
during coaching visits, heightened the experience: 
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“These big, big, top, top, top, people were here. It’s not 
the normal people like we that they (the staff] are used 
to. So that one alone will give them some 
inspiration...” (DHMT member) 


Since the teams had little formalised management 
training, the novelty of the LDP disposed them to being 
receptive to capacity support. The exposure to manage- 
ment practices enabled teams to attain their LDP results, 
and they noted that the imposition of deadlines created 
a sense of urgency and increased the need to attain re- 
sults, compared to their routine targets. The LDP also 
helped managers build initiative. Managers acknowl- 
edged that some problems were ‘beyond’ them, and 
therefore, initiative-taking was encouraged, but only on 
a ‘small-scale’: 


“You are supposed to make do with what you have. 
Because sometimes when we have challenges we think 
that ‘oh as for this one, we are waiting for region to 
come and do it, or we are waiting for national to come 
and do it. LDP says you shouldn't think so big, but 
something within... you should just try to think around 
yourself’. (DHMT member) 


Managers learned to better prioritise and felt more 
able to manage concurrent programmes, and_ thus 
viewed themselves as working more efficiently. Support- 
ing teamwork through inspiration and acknowledgment 
was also important. One manager stated that prior to 
the LDP she used to ignore her staff if they incorrectly 
performed a task. Managers did note that the LDP had 
no influence on the relationship between district and re- 
gional levels, nor did it alter the dynamics around 
resources: 


“It hasn't changed our resources. If I am saying the 
truth, I don’t think we have the resources to work 
with”. (Sub-district head) 


One issue reported consistently by district managers 
was the LDP’s resource intensiveness. Convening stake- 
holders and running training workshops all require add- 
itional funds, which was perceived as burdensome, since 
teams had severe resource constraints and had not bud- 
geted for the LDP in advance. Lobbying for funds from 
the District Assembly was difficult as the annual plan- 
ning cycle had already passed. Furthermore, the time re- 
quired to meet for LDP activities, convening the wider 
team, and preparing plans was viewed as onerous in the 
face of concurrent programmes and other health system 
constraints. One manager highlighted the difficulty that 
under-staffing created in trying to gather staff for train- 
ing without disrupting service delivery: 


“The challenges that we had in implementing the LDP 
were trying to get staff themselves to come around to 
listen to us. It’s terrible, the beginning it was very hard 
to get the unit heads to come around. The reason was 
that due to lack of staff. The unit heads must be there 
to monitor, and there is no staff to bring to come and 
listen to us.” (Hospital management) 


Managers widely perceived the lack of LDP institutio- 
nalisation as related to the LDP being a ‘regional 
project’: 


“You can also see that at the regional level it has 
ended. So if the regional level it has ended can the 
district continue? Since then there has never been any 
coach from region to come and see what we have done, 
where we have reached and what the challenges are. 
So you can imagine, we at the sub-district can we also 
do it? So me, it is not about the district not doing it or 
it's not implementing it, I only see it as a project...at 
the {LDP workshops] we were told it is not a project, 
it is a running thing. But it has ended as if it is a 
project and the project has come to an end.” (DHMT 
member) 


Proposing causal linkages (C+ M-> O) 

We illustrate the relationships between our contexts, 
mechanisms, and outcomes in a CLD* (Figure 2). This 
schema represents causation between variables, with di- 
rections of influence depicted by arrows. Influence in 
the same direction is represented by positive arrows. 
Feedback loops can reinforce (R) or self-regulate the 
pathway [32]. 

Our CLD shows the pathway of the LDP’s short-term 
outcomes (right-side of the figure, thin arrows) and 
medium-term outcomes (left-side of the figure, thick ar- 
rows). On the short-term, the novelty of the LDP for man- 
agers with limited formalised management training (C) 
stimulated the value and utility of bundled management 
practices taught by the LDP (M) for teams to achieve their 
LDP results (O). This causal pathway is linear, and does not 
significantly deviate from the predicted programme theory 
of the LDP. On medium-term outcomes, the introduction 
and facilitation of the LDP in a top-down manner (i.e., from 
the region) (C) promoted hierarchical authority and trig- 
gered the LDP’s focus on controls and standardisation (M). 
Multiple, reinforcing feedback mechanisms (Ri and R2) 
neither supported LDP institutionalisation, nor systems 
thinking among district teams (O). 

Had our original assumption been borne out, we 
would see a third reinforcing loop (R3) between systems 
thinking and LDP institutionalisation. For simplicity 
sake, we redraw the same analysis as a causal tree dia- 
gram (Figure 3). 
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Discussion 

Hawe et al. [30] suggest that the most important dimen- 
sion of complexity is frequently not the complex inter- 
vention itself, but rather the context into which it is 
introduced. We found that in trying to produce change 
in a complex adaptive system, the LDP in this case could 
not be sufficiently institutionalised. In essence, the sys- 
tem ‘rejected’ it and returned to its prior equilibrium. 
The context of system hierarchy, as demonstrated by the 
deployment of regional staff to train the districts, high- 
lights the cascading approach to systems change from 
the top. This may not always be appropriate, and further 
underscores the need to think systemically when intro- 
ducing any intervention. We note that, in this case, the 
LDP appeared to engage systems thinking in its tools ra- 
ther than through its practices, and incorporated its CQI 
elements in its organisational outcomes rather than its 
processes. This suggests a focus on organisational con- 
trol, rather than creativity, of both the LDP and the or- 
ganisational context into which it was introduced. Being 
tool-driven, the LDP does not itself provide processes 
for developing a learning organisation, and we noted no 
evidence of new mental models created in district teams 
~ what Sterman [33] distinguishes as ‘single-loop’ versus 
‘double-loop’ learning. Our study raises questions about 
the nature of management and leadership capacity 
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Figure 2 Causal loop diagram of LDP implementation, February to August 2012. 
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strengthening. We recognise that short-term capacity 
strengthening interventions may not necessarily support 
such reorientations. As such, it is critical for donor part- 
ners and national governments to reconsider the types 
of idealised interventions often put in place, and how 
contexts can modulate expected outcomes over time. 
This suggests support for longer term, more reflective, 
and potentially unpredictable capacity strengthening ap- 
proaches. This notion is further supported by a recent 
study from Rwanda that found no statistical association 
between training and adherence to recommended MNH 
practise [34]. Our findings uphold earlier work by Blaise 
and Kegels [35], who describe the rigidity and lack of re- 
Sponsiveness in command-and-control structures ob- 
served in several African health systems as contributors 
to quality of care challenges in service delivery. 

At the outset we hypothesised that, in reinforcing sys- 
tem hierarchy, the LDP’s underlying goal of organisa- 
tional creativity would prevail due to the context of high 
uncertainty. Paradoxically, the LDP’s underlying goal of 
organisational control was more pronounced. We attri- 
bute this to the degree of centralised decision-making in 
the system: the strength of ‘command-and-control’ over- 
rides other mechanisms that enable learning, creativity, 
and adaptability. This mismatch in contextual uncer- 
tainty and organisational culture may very well account 
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for the lack of effective management at district level. 
With this in mind, we refine our MRT as follows: 

The LDP brings about its short-term outcomes through 
its experience of novelty, building initiative, supporting bet- 
ter prioritisation, and building teamwork. The LDP rein- 
forces hierarchical authority due to being introduced in a 
top-down manner. As such, resource uncertainty remains 
high and, as a consequence, district manager decision- 
space remains narrow. Thus, district managers continue to 
rely on trust and respect as coping mechanisms to deal 
with resource uncertainty and their managerial risk. The 
context of high uncertainty, coupled with reinforced hier- 
archical authority, triggers the LDP’s underlying focus on 
organisational control. Systems thinking is not stimulated, 
and LDP institutionalisation does not occur. 

In thinking about how the LDP might have been im- 
plemented differently, we consider five ways in which 
the causal pathway could have been altered: i) had the 
LDP facilitators been peers instead of superiors (for ex- 
ample, training teams could have consisted of district 
managers whose districts had previously undertaken the 
LDP rather than being regional officers), this may have 
weakened hierarchical authority, thereby reducing the 
top-down nature of its introduction; ii) had districts 
volunteered to receive the LDP instead of being ran- 
domly selected, they may have expected it and better 
prepared their resources; iii) had ongoing mentorship 
and coaching been built into the process through sys- 
tematic follow-up, this may have supported the view of 
greater district ownership; iv) had the timeframe of the 
LDP intervention been lengthened to include two or 
three cycles, this may have had longer-lasting effects and 
become routine practise; and v) had organisational cre- 
ativity and learning been an explicit goal, with reflective 
processes as a major part of the intervention, this may 
have provided greater opportunity for more systems 
thinking to develop in district managers. We recom- 

mend that the LDP could be strengthened by a more 
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Figure 3 Causal tree diagram of LDP implementation, February to August 2012. 
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explicit integration of CQI philosophy and principles 
into its existing tools, and greater attention paid to 
context to support its institutionalisation. We are aware 
of existing CQI-based interventions in the Ghanaian 
health system with similar ‘Plan-Do-Study-Act’ cycles, 
indicating that the lack of institutionalisation of one 
programme does not prevent the implementation of 
other similar interventions. 

Our findings clearly demonstrate that a lack of consid- 
eration of the context into which such interventions are 
introduced can minimise their effectiveness. More im- 
portantly, our work highlights the fact that context also 
informs the kind of management and leadership that 
emerges at district level. Not uniquely a Ghanaian chal- 
lenge, decision processes are often rooted in a desire for 
control and prediction, such that managers who cannot 
deliver are perceived as ineffective and are soon replaced 
[36]. These issues exceed the scope of our study, but do 
underscore the fact that improvements in management 
and leadership do not reside in the capacities of man- 
agers alone, but demand keen attention to the organisa- 
tional contexts in which managers are embedded. 

A limitation of our study is that it reports on only one 
context for LDP implementation. This is a first level 
analysis; moving forward we expect to conduct a wider 
exploration of other districts in the Greater Accra Re- 
gion and further refine our MRT. 


Conclusions 

The influence of contexts on mechanisms in the gap be- 
tween short- and medium-term outcomes is particularly 
important given that decisions to scale-up interventions are 
frequently based on their success in the short term. In the 
Ghanaian context, introducing the LDP into a context of 
highly centralised decision-making and resource uncer- 
tainty triggered its underlying goal of organisational control. 
More explicit focus on systems thinking principles that en- 
able district managers to better cope with their contexts 
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may strengthen the institutionalisation of the LDP in the 
future. 


Endnotes 

“The Ghana Health Service (GHS) is the semi- 
autonomous agency of the Ministry of Health respon- 
sible for public health service delivery. Since its creation 
in 1996, the GHS has been administratively decentra- 
lised along national, regional, and_ district lines. 
National-level GHS is responsible for policy direction. 
Regional health directorates provide technical and ad- 
ministrative oversight to district health directorates re- 
sponsible for coordinating service provision at district 
and sub-district level. 

>This was as part of a larger study to improve service de- 
livery for MNH. One key aspect of the study sought to sup- 
port managerial decision-making as part of the overall 
organisational change needed to improve MNH quality in 
the Greater Accra Region. The larger study envisioned de- 
signing and implementing a CQI programme to address 
this. However, given the existence of the LDP already in the 
health system, and its overlap in approaches and philosophy 
with CQI, the LDP was instead identified as the CQI-based 
intervention to be evaluated. 

“Dangme West was subsequently divided into two sep- 
arate districts: Shai-Osudoku District and Ningo- 
Prampram District; this paper reports its findings against 
the district structure at the time of the study. 

“We drew Figures 2 and 3 using Vensim simulation 
software : www.vensim.com. 
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Abstract 


Background: New forms of leadership are required to bring about the fundamental health system changes 
demanded by primary health care (PHC). Using theory about complex adaptive systems and policy implementation, 
this paper considers how actors’ sensemaking and the exercise of discretionary power currently combine to 
challenge PHC re-orientation in the South African health system; and provides examples of leadership practices that 
promote sensemaking and power use in support of PHC. 


Methods: The paper draws on observational, interview, and reflective data collected as part of the District 
Innovation and Action Learning for Health Systems Development (DIALHS) project being implemented in Cape 
Town, South Africa. Undertaken collaboratively between health managers and researchers, the project is 
implemented through cycles of action-learning, including systematic reflection and synthesis. It includes a particular 
focus on how local health managers can better support front line facility managers in strengthening PHC. 


Results: The results illuminate how the collective understandings of staff working at the primary level - of their 
working environment and changes within it — act as a barrier to centrally-led initiatives to strengthen PHC. Staff 
often fail to take ownership of such initiatives and experience them as disempowering. Local area managers, 
located between the centre and the service frontline, have a vital role to play in providing a leadership of 
sensemaking to mediate these challenges. Founded on personal values, such leadership entails, for example, efforts 
to nurture PHC-aligned values and mind-sets among staff, build relationships and support the development of 
shared meanings about change; instil a culture of collective inquiry and mutual accountability; and role-mode! 
Management practices, including using language to signal meaning. 


Conclusions: PHC will only become a lived reality within the South African health system when frontline staff are 
able to make sense of policy intentions and incorporate them into their everyday routines and practices. This 
requires a leadership of sensemaking that enables front line staff to exercise their collective discretionary power in 
strengthening PHC. We hope this theoretically-framed analysis of one set of experiences stimulates wider thinking 
about the leadership needed to sustain primary health care in other settings. 
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Background 

Reform and renewal are fundamental features of every 
health system, though the ambition and scale of change 
varies over time and between countries. Twenty years 
after the election of its first democratic government, 
South Africa continues to strive for an improved health 
system — a health system that better meets the needs 
and preferences for treatment, care, and dignity, of all its 
population. The fragmented health system inherited from 
the previous era, with multiple organizational structures, 
levels, and programmes, was shaped by the perverse polit- 
ical and economic goals of the Apartheid state [1]. Various 
policy, organizational, and resource allocation reforms 
have been implemented since 1994 to re-orient the system 
towards population health need and equity goals. None- 
theless, recent reviews have highlighted slow progress, 
particularly in establishing a functional district health sys- 
tem (DHS) as a basis for strengthening primary health 
care (PHC) [2,3]. Towards Universal Health Coverage, 
and in line with global policy directions [4,5], South Africa 
has, therefore, placed renewed urgency on PHC and DHS 
development [6-8]. 

International experience shows that re-orienting health 
systems towards PHC challenges existing ways of work- 
ing [4,9]. In South Africa, dispersed accountability, com- 
plex rules and procedures, and an organizational culture 
of deference to hierarchy within it also “overwhelm ra- 
tional policy debate and the implementation of new pol- 
icy’ [10]. As a result, and as pointed out by the National 
Department of Health, the South African health system 
remains strongly hospi-centric and specialized, with 
decision-making driven more by service than population 
needs [6]. As elsewhere, the pro-active pursuit of popu- 
lation health needs and equity goals in PHC strengthen- 
ing requires, therefore, fundamental changes in the way 
health system actors think and work, in its organizational 
culture, supported by new forms of health system leader- 
ship [11,12]. Although there is only limited evidence about 
what such leadership entails, theoretical perspectives sug- 
gest that ‘reculturing an organisation’ involves empower- 
ing front line workers to think and work differently by 
encouraging subtle change in the values, customs, rela- 
tionships, and conversations shaping their behaviour 
[13-15]. 

In this paper, we present experience to illuminate both 
the challenges that confront efforts to strengthen PHC 
within the South African health system, and the nature 
of leadership needed to mediate such organizational 
change. Our analysis is framed by the concepts of sense- 
making and discretionary power, drawn from theory 
on complex adaptive systems and policy implementation, 
respectively. We argue that, to become a lived reality 
within the DHS, those working to support primary 
and community-based services, including PHC facility 
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managers and their staff, must be able to make sense of 
PHC-promoting policies and plans, and incorporate 
them into their everyday routines and practices. This re- 
quires new forms of leadership by the health system’s 
middle managers, namely sub-district managers: a lead- 
ership of sensemaking in support of PHC strengthening. 

We are a team of health system managers and re- 
searchers working together to understand and act in the 
district health system, through cycles of collaborative ac- 
tion and learning, in Mitchell’s Plain health sub-district, 
Cape Town, South Africa. Our analysis represents a 
theoretically-framed reading of one particular set of ex- 
periences in one particular place, generated through a 
careful, systematic, and reflective research collaboration. 
We do not seek to derive discrete policy lessons about 
particular activities that can strengthen PHC in South 
Africa or elsewhere. Instead, recognizing policy learning 
as an organic process [16], our intention is to stimulate 
those working in other settings to think differently about 
the forms of leadership needed to sustain PHC. 


Methods 

Study approach and focus, data collection, and data 
analysis 

The experiences we present are drawn from the District 
Innovation and Action Learning for Health Systems 
Development (DIALHS) project, initiated in 2010 as a 
service-research partnership focused on governance is- 
sues within the Mitchell’s Plain health sub-district, Cape 
Town. It involves collaboration between two health au- 
thorities (the City of Cape Town and the provincial 
health department of the Western Cape government), 
and two universities (the Universities of Cape Town and 
the Western Cape). 

We began our engagement by conducting a situation 
analysis to understand the managerial structures and 
processes of the sub-district and its location in broader 
district functioning. Discussion of this analysis then led 
us to consider further how sub-district managers can 
better support PHC facility managers to lead their staff 
teams. As Figure 1 indicates, we have subsequently ad- 
dressed this concern through iterative cycles of collab- 
orative action and learning [17,18], including a focus on 
facility managers’ experience, that have entailed cycles 
of data collection, analysis, and interpretation. Reflect- 
ive practice has been a common approach in all our ac- 
tivities, and is itself an intervention in managerial 
practice [19]. 

Table 1 outlines the range of data generated across our 
activities, which include transcripts and notes of general 
reflective discussions within the research team, and with 
managers. This paper is based specifically on the internal 
report of the initial situation analysis (2010), notes of 
key DIALHS discussions within sub-district management 
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Activities: 

1. Community profiling follow-up 

2. Understanding information & 
knowledge use by PHC facility 
managers 


3. Leadership development suppor; 


Activities: 

1. Community profiling follow- 
up 

2. Understanding PHC facility 
managers’ identity transition 

3. Group coaching support for 
PHC facility managers 


Future years 


Situation analysis: 


mapping governance 
structures, processes, 
challenges & 
opportunities 


Activities: 

1. Community profiling to role-model 
population orientation 

2. Understanding PHC facility managers’ 
identity transition 


Figure 1 The DIALHS cycles of collaborative action and learning. 


meetings (2010-2012), researcher field notes (2010- 
2012), transcripts and notes of 22 interviews and personal 
meetings with managers at district and sub-district level 
(2010-2013), and transcripts of 5 research team reflective 
discussions (2011-2012). Following the principles of the- 
matic analysis, these data were initially systematically 
reviewed with our conceptual starting points in mind (see 
below), by the researcher-authors (LG, UL). An initial 


Activities and engagements which generated data 


synthesis and narrative of experience was then developed 
for discussion with the other authors, the two primary 
sub-district managers (SE, PO), in a validation process 
equivalent to member and peer-checking that also gener- 
ated further insights drawn into the final paper. Therefore, 
in our work, we have been systematic, a core criterion 
of validity in action learning [17], as in other research 
[20]. We have also allowed double-loop learning [21], 


Situational analysis 


Cycles of 


Planning interventions 

Community profiling and local area groups 
Support for environmental health practitioners 
Support for PHC facility managers 

HIV/AIDS & TB programme roles 
Implementing of interventions 


Review and reflection 


Research sub-studies 


The transition process from nurse to facility manager 


The information used by facility managers in routine 
decision-making 


Meetings and reflections of research team 


Interviews and reflective conversations with sub-district and 
district managers 


Notes of meetings with sub-district managers 


Review of policy documents and minutes of statutory meetings 
Stakeholder interviews 
Observations of meetings 


Presentations and meeting/workshop notes; document reviews Composite reports 


Presentations, notes of meetings, field notes, and reports 
Notes of meetings with teams involved in intervention 


Presentations and reports to ISDMT meetings 


Interviews 


Observations 


Transcriptions and notes of reflective meetings of research team 


Notes of meetings with district managers 
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deepening understanding of experience through reflection 
during our work and in developing this paper — stepping 
back from the initial narrative, interrogating our assump- 
tions, and viewing it from different perspectives. 

Our work has ethical clearance from the Human 
Research Ethics Committee of the Faculty of Health 
Sciences, University of Cape Town (Ref 039/2010), and 
research approval from both the City of Cape Town 
and Western Cape Provincial Government Department 
of Health. 


Conceptual foundations 

Sensemaking can be understood as “the process individ- 
uals undertake as they try to understand what is going 
on around them, as they try to make sense of events and 
experiences” |22]. In sensemaking, our mental models, 
that is our beliefs and assumptions about how the world 
works [23,24], help us to notice phenomena in our en- 
vironment, which we then categorise and label, making 
meaning of them, ultimately as a basis for acting. Sense- 
making is, therefore, about the interplay of interpret- 
ation and action [25-27], 

The adaptive agents within complex adaptive sys- 
tems (CAS) are sensemakers, whose interpretations of 
their world are shaped by the system paradigm, the 
underpinning, often unspoken but shared, social agree- 
ments about the nature of reality in that system [23]. 
Because agents in every system are interconnected and 
interdependent, their many daily interactions also re- 
sult in the emergence of shared ways of being and 
doing, patterns of collective behaviour that are taken 
for granted [28]. These system structures, the manner in 
which a system’s elements are organized, include, for 
example, practices of inter-personal engagement and in- 
formation flows; they, in turn, shape the patterns of 
organizational life that generate the events that we most 
easily notice [241]. 

Although not recognized as CAS theory, Lipsky’s 
[29,30] theory of Street Level Bureaucracy (SLB) illumi- 
nates CAS ideas with specific reference to public policy 
implementation. The discretionary power of ‘street level; 
or front line, workers in public bureaucracies exists be- 
cause they are “free to make a choice among possible 
courses of action and inaction” |31| within the rules 
shaping their behaviour, allowing them to translate pol- 
icy through their practices and interactions with clients. 
Lipsky argued that as they interpret, choose, and act, 
they are guided by the mental models they develop to 
manage their demanding work settings characterized by 
heavy workloads, resource constraints, and centrally di- 
rected and often unclear, policy imperatives. In these set- 
tings the unanticipated consequences of the ways they 
manage their time and engage with their clients include 
limiting access to public services and treating clients 
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disrespectfully. However, front line workers can also en- 
gage positively with clients, particularly when encour- 
aged to use their discretionary power to be responsive to 
clients [32]. 

These bodies of theory both suggest that hierarchical, 
command, and control leadership practices do not take 
account of the reality of complex adaptive systems and 
policy implementation. SLB theory specifically notes that 
top-down action to control the use of discretionary power 
will only encourage front line workers to stereotype and 
disregard client needs [29]. The sensemaking literature 
Suggests, moreover, that during times of organizational 
change individuals try to make sense of their experience 
by engaging with others, generating shared interpretations 
that, in turn, shape their behaviours and trigger further 
sensemaking. This cycle generates new, shared ways of 
working that may not be aligned with the intentions of 
new initiatives [26]. The theory suggests, therefore, that 
particular forms of leadership are required to implement 
policy and bring about organizational change in CAS. 
Such leadership needs to be distributed across all levels of 
an organization, placing particular demands on middle- 
level managers [33]. Moreover, as “real leverage exists deep 
in the recesses of the systems — mind-sets, values, beliefs — 
where identify is created” [28], such leadership needs to 
mediate sensemaking and support changes in the shared 
assumptions about how people should act (so influencing 
their exercise their discretionary power) in different situa- 
tions [23,24]. 


Results 

Drawing on our conceptual foundations we now present 
experience from a health sub-district in Cape Town, 
considering both sensemaking and exercises of discre- 
tionary power, and leadership practices that seek to 
recognize both in supporting new ways of working. We 
start by describing the setting of our work. 


Mitchell’s Plain health sub-district 

In 1976, residents from 250 different communities across 
Cape Town were forcibly removed and settled in Mitchell’s 
Plain, when it was designated a ‘separate’ area for so-called 
‘coloured people’. This birth in violent social dislocation 
reverberates to this day, and it is now one of the poorest 
areas of the city. High unemployment and low labour ab- 
sorption (at 24% and 46%, respectively, according to the 
2011 census) [34], as well as substance abuse and poor 
schooling, exacerbated by massive population growth, 
contribute to vicious cycles of poverty, crime, and social 
destabilization. 

Public PHC services in Mitchell’s Plain health sub- 
district are provided to a population of over 510,000 
(2011/2012) by both local and provincial government fa- 
cilities focused, respectively, on preventive, promotive, 
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and curative child health services, and adult curative 
care.” Indicators point to a relatively strong public health 
service performance in the sub-district: a tuberculosis 
(TB) cure rate of 88%, immunization coverage at 93%, 
and 56% of ante-natal care visits occurring before 
20 weeks. However, there are still numerous under- 
served communities, particularly in the newer, fast de- 
veloping areas of the sub-district; and there are regular 
patient complaints about poor quality of care and areas 
of service delivery weakness. Key health problems in- 
clude a co-infection rate of TB and HIV of 50%, non- 
communicable diseases, mental ill-health, and violence 
({(35], and data from 2011/2012 District Health Expend- 
iture Review for Cape Town). 

Efforts to integrate and strengthen PHC service provision 
across the two health authorities over the last 20 years have 
been complicated and hampered by a range of legal, labour, 
and financial obstacles. The 2008 establishment of the 
Metro District Health System (MDHS), as part of the pro- 
vincial government's wider vision of strengthening PHC,” 
provided the structural platform for delivering comprehen- 
sive and integrated PHC services in Cape Town. At 
present, the Mitchell’s Plain integrated sub-district man- 
agement team (ISDMT) coordinates service delivery in 
agreed areas between local (City of Cape Town, CoCT) 
and provincial government (MDHS). 

The two Mitchell’s Plain health managers were appointed 
to their current posts in 2005 (SE, CoCT sub-district 
manager) and 2012 (PO, MDHS sub-structure manager). 
Their position within the complex lines of authority that 
make up the Cape Town health system is shown in 
Figure 2. Located at the interface between top-down stra- 
tegic planning processes and bottom-up operational 
decision-making processes and action, they are the middle 
managers [36] tasked with leading the establishment of a 
health system oriented towards population health needs 
as envisioned by provincial health policy (manager inter- 
view notes, 1 July 2010). Working within centrally-set 
budget limits and human resource management guide- 
lines, they both have some decision-making latitude. 
Overall, they are responsible for the management of re- 
sources, people and perceptions (manager interview notes, 
19th July 2010), and judge that their biggest challenges lie 
in managing people and their perceptions [35]. 

The sub-district health managers see PHC facility man- 
agers as key players in strengthening PHC in Mitchell’s 
Plain. These managers and their staff represent the face of 
the health system to the public, are responsible for its re- 
sponsiveness to people and patients, and should be among 
the first to pick up community health needs and concerns. 


Mind-set challenges to PHC in Mitchell’s Plain 
Over the last 20 years, PHC in Mitchell’s Plain has be- 
come “a more complex environment for those working in 
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and for the community. Nurses see a lot of sick children 
and sometimes the children die. Nurses don’t all necessar- 
ily have the right skills (for these situations, and given high 
staff turnover). So facility managers have to manage this 
complexity and also don’t all have the skills. And they 
have little confidence in the system — for example, ambu- 
lance services don’t arrive on time. And the managers 
don’t know how to talk to the community for example over 
deaths in facilities, or how to help staff cope with the 
demands” (Manager interview notes, 13th April 2012). 

In this environment, there is a duality of sensemaking 
and action. As a group, front line staff often seem to 
end up working against PHC-oriented change, even 
though as individuals they might support its goals. Fa- 
cility managers and their staff commonly maintain col- 
lective mind-sets that are steeped in the autocratic and 
procedural cultures of a previous era and that run 
counter to a population health and PHC orientation. 
The sub-district managers note that some staff display 
rather “authoritarian and autocratic attitudes towards 
patients, and do not want to share power with them” 
(ISDMT notes, 19th Jan 2011). They also think that fa- 
cility managers do not always “understand the ‘big pic- 
ture’ of facility services, that is, the health needs of the 
population they serve, the challenges patients face in 
in accessing services, and so the importance of new 
community-based activities and programmes. They don't 
generate and use information to understand the needs of 
the population they serve, and ‘they don’t think beyond the 
people coming into the facility’ (Researcher field notes, 
6th July 2012). Indeed, as the PHC facility managers are 
all professional nurses by training and receive limited 
support when first appointed, many of them prefer, and 
even feel more confident, in their clinical rather than 
managerial roles [37]. 

Often, facility managers also seem to work in a very 
procedural way — adopting an attitude that says, “I want 
the piece of paper that tells me how to act’ (Research 
team notes, 14th December 2010). Whilst recognizing 
that these managers are mostly quite capable and com- 
petent, sub-district managers often see them as having 
an “external locus of control” —being too passive, not 
persistent in dealing with problems in their facilities, in- 
cluding complaints from patients, or in keeping track of 
their own performance (Manager interview notes, vari- 
ous dates; Research meeting notes, 4th December 2012). 
Timekeeping and keeping up with routine schedules of 
activities is also judged to be poor. When given new 
decision-making authority, facility managers appear to 
respond fearfully rather than by embracing the oppor- 
tunity: it seems as if “they don’t take responsibility for 
new activities or actions to improve services. They say to 
their staff, ‘the sub-district manager said you must do it: 
rather than projecting a positive image of the activity 
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and owning it” (Researcher field notes, 6th July 2012). 
Such resistance is typical of the exercise of discretionary 
power by street level bureaucrats [29], but also reflects 
‘change fatigue’ [38-40] and organizational uncertainty 
after nearly 20 years of constant health system change. 
The continuing debates about if, when, and how local and 
provincial government PHC services will be integrated in 
Cape Town have only aggravated this uncertainty. 

The retention of paternalistic and autocratic approaches 
is, moreover, not restricted to frontline workers. Staff 
survey results indicate that communication practices are 
commonly perceived to be quite hierarchical in the wider 
health system [41]. Some sub-district managers even talk 
about PHC facility managers as “their children’, express- 
ing the need “to watch over them” (Research meeting 
notes, 4th December 2012). An ‘ambivalence towards au- 
thority’ among South Africa civil servants is, moreover, a 
general apartheid legacy [42] that is compounded, within 
the health system, by “the accumulated weight of existing 
practices and procedures, together with embedded hier- 
archies that institutionalise a specific distribution of power 
and privilege” {10}. 


Sensemaking and resistance to centrally-led PHC 
improvement targets 

In strengthening PHC, the experience around annual targets 
provides an important example of how sensemaking plays 
out, given dominant mind-sets. In line with provincial and 
local government health plans, annual targets are centrally 
established within strategic planning processes to drive PHC 
service improvements in Cape Town (e.g., for tuberculosis 
cure rates or extension of basic ante-natal care across facil- 
ities). These targets are backed up by regular monitoring 
through ‘plan, do, review’ (PDR) meetings where managers 
at different levels come together to examine facility, sub- 
district and district performance against targets, identify 
challenges and develop actions to address them. 

Mid and senior health managers see these processes as 
“providing standardized frameworks to guide lower level 
managers and, more specifically, providers, to work differ- 
ently to better meet population health needs” (Manager 
interview notes, 19th July 2010). They also argue that 
“policy provides a stable structure within which people 
know what is expected of them’ (Manager interview notes, 
13th April 2012) and that standardization higher up the 
system is “to give some predictabilityNogic to allow for 
innovation lower down... to bring the certainty needed for 
innovation lower down the system” (Manager interview 
notes, 19th July 2010). 

However, both positive and negative meanings have 
become attached to the word ‘targets’ in the Mitchell’s 
Plain health system discourse. The positive potential is 
expressed as “directing people towards common goals, or 
giving people a motivating force”. Reaching a target can, 
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thus, bring a sense of achievement and positive energy 
(Research meeting notes, 4th December 2012). In con- 
trast, the dominant language PHC facility managers and 
staff use around targets is quite negative — with targets 
seen as disempowering, as a disciplinary tool, and as en- 
couraging or enabling micro-management by higher-level 
managers. Perhaps refracted through the prism of history 
and wider organizational culture, facility managers seem 
to understand the word ‘targets’ as authoritarian and 
therefore illegitimate: “/t’s all that is bad in the system... it 
also says ‘we don’t have agency...‘we are bombarded, can’t 
do anything else; so it removes accountability and responsi- 
bility for anything other than the target’ and so “a lot 
of the target conversation is completely disembodied, 
its removed from the actual meeting of service needs” 
(Research meeting notes, 5th December 2012). 

Given the prevailing mental models, targets and the 
PDR processes are, in a sense, filtered through a power 
battle between managers seeking to give clear and consist- 
ent direction for strengthening PHC to multiple actors, 
and front line staff, shaped by histories and cultures of 
passivity and dominance, resisting change and afraid to 
take on new responsibilities. Although facility managers 
mostly comply with reporting requirements, they have not 
yet adopted the broader problem-solving attitude or will- 
ingness to take ownership of, and make effective, efforts 
to improve PHC services. As noted in other settings 
[29,43], central efforts, such as target setting, that seek 
both to contain the discretionary power of street level bu- 
reaucrats and direct it towards imposed goals often have 
unexpected consequences. Planned (imposed) change may 
encourage compliance without conviction [44], because, 
as sensemaking theory notes, it fails to provide spaces for 
the new forms of sensemaking necessary to support the 
intended changes [26]. Ultimately, therefore “change is it- 
self an interpretive process” in which “[t]he meaning of top 
down initiatives emerges bottom up” {33}. 


Supporting PHC strengthening through a ‘leadership of 
sensemaking’ 

From our first engagements, sub-district managers recog- 
nized the challenge of having to address the passivity of 
PHC facility Managers and their staff, and their role in 
empowering them “to work differently to meet needs”, as 
critical issues in Mitchell’s Plain (Manager interview notes, 
19th July 2010). Within the DIALHS collaboration we have 
subsequently, in a mixture of deliberate and spontaneous 
actions, tried out different ways of engaging and support- 
ing PHC facility managers to recognize and address popu- 
lation health needs, working with their staff. Our initiatives 
include both new, joint activities (e.g., the community pro- 
filing initiative) and adaptations to routine processes and 
practices (e.g., the key performance area (KPA) process). All 
seek to encourage PHC facility managers to take ownership 
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of their own performance, and that of their staff, as well as 
to hold them to account for it (Manager interview notes 
1Sth Oct 2012; 7th December 2012; 9th April 2013). 


The community profiling initiative (generating 
local knowledge, priorities, and action through 
multi-actor engagements) 


In early 2011, we initiated an activity that aimed to 
strengthen three inter-related planning and management 
priorities in the sub-district: i) ‘shifting the lens’ of 
service providers from a patient to a stronger population 
orientation in health system organization and functioning, 
as advocated by provincial and national policy 
guidelines; ii) moving the sub-district’s thinking and 
visioning beyond one-year planning cycles; and iii) 
strengthening relationships between service providers 
and community representatives. 

Provincial and district management had recently 
emphasized the need to address health of populations, 
rather than patients alone, in their policy and 
planning guidelines. Yet, facility managers had 
repeatedly expressed uncertainty and frustration with 
the fact that they “did not know the communities they 
were serving’, did not know how to engage with other 
role players or access other health resources in 
communities, and were overwhelmed with the need to 
service short-term targets instead of being responsive 
to local priorities and needs. 

A series of larger and smaller workshops brought together 
multiple stakeholders from health authorities and civil 
society to identify and map health resources and gaps, as 
well as to identify appropriate local action and planning 
priorities to address the gaps and challenges. 

These activities succeeded in overcoming or at least 
lowering perceived barriers between different groups 
and brought actors into conversation with each other, 
drew on their shared informal knowledge of local 
health contexts, and provided opportunities to 
generate shared meanings about those contexts. 

These conversations’ have subsequently led to some 
specific health initiatives, such as continued action to 
share knowledge among groups and tackle 
environment health problems in certain communities. 
However, maintaining local area groups in all areas in 
the sub-district has proved difficult. 


The local government PHC facility manager ‘KPA 
process’ (developing local service improvement 
priorities) 


In 2010, the local government sub-district health 
manager introduced a new process to encourage 
pro-active planning and action by facility managers, 
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involving: i) setting clear, locally appropriate objectives 


within the broad priorities specified in established 


health plans; ii) outlining activities, intended outcomes 


and monitoring and evaluation approaches; and iii) 


holding facility managers to account for implementing 


agreed actions. 
Working with support, facility managers each 


developed their own KPAs and then presented them to 


the whole sub-district managerial team in late 2010. 
During the course of 2011 they periodically reported 
back on progress in implementing agreed actions 

and in late 2011 developed a new set of KPAs for 
2012. In parallel, the routine PDR meeting between 
facility managers and their line managers in which 
facility challenges are discussed, was re-named and 
re-structured to allow a stronger regular focus on 
collectively considering how to address common PHC 
facility challenges, including sharing ‘best practices’ 
and success stories among these managers. Using 
existing language and the KPA terminology to 
introduce the new process, the sub-district manager, 
nonetheless, reframed this language to emphasize its 
developmental and sense-giving potential. She also 
role-modelled constructive accountability through 
creating a space to allow collective consideration of 
challenges and successes. 

After two years, implementation is uneven, as follow-up 
and consistent documentation have been lacking. While 
some managers easily saw and acted on the opportunity 
to self-determine priorities, others will need more 
support to gain the confidence and skills to identify 

and act on local priorities. 


Reflecting on our activities through theoretical lenses 
throws light on five possible elements of a ‘leadership 
of sensemaking’ for PHC strengthening. The import- 
ance of middle managers’ personal values as a founda- 
tion for other leadership action is the first element [14]. 
Leadership values and capabilities of particular rele- 
vance to sensemaking for PHC might include concern 
for the population being served and the broader social 
determinants of health; recognizing the potential in 
other people, for example by adopting a mentoring ap- 
proach towards other staff; and being reflective and 
self-critical — willing to learn and change one’s own 
practices (Manager interview notes, 13th April 2012; 
9th April 2013; 20th May 2013). 

From this foundation, we have applied four other 
cross-cutting leadership practices in supporting PHC 
facility managers: 


i. Nurturing the values and moral purpose of PHC staff 


ii. Building relationships to support the development of 
shared meanings about change 
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iii. Instilling a culture of collective inquiry and mutual 
accountability within the sub-district 

iv. Role modelling critical management practices and 
using language to signal new meanings 


First, nurturing the values and moral purpose of PHC 
staff — encouraging facility managers, for example, ‘to 
dream’ about working differently ISDMT meeting notes, 
20th October 2010). 

Re-orienting front line health staff towards a popula- 
tion health focus, encouraging them to take a pro-active 
role in managing services to meet community need, re- 
quires a “real mind-shift for managers and staff’. “A 
community orientation has to become part of people’s 
way of being’ (ISDMT meeting notes, 19th Jan 2011), 
but some facility managers and staff do not currently 
have, and even resist, this orientation. The sub-district 
managers, therefore, constantly and consistently affirm 
the importance of patients and the broader population 
in all their engagements with staff — for example, en- 
couraging facility managers to align broader goals and 
targets with local priorities, or to respond speedily to pa- 
tient complaints. 

Within the DIALHS collaboration, we also, more for- 
mally, initiated a collaborative community engagement 
process in 2011 to encourage conversation about local 
health needs and resources among different stakeholders 
in the sub-district (see Community profiling initiative). 
We expressly framed this activity within the context of 
the social determinants of health and allowed facility 
managers to think about the world outside their facil- 
ities. In implementing this process, sub-district man- 
agers noted that it was important to role model new 
mind-sets and use new language: “whoever looks at us 
needs to know that as a team we are committed to the 
DHS and PHC, and building it with a population focus 
and orientation— and this is what we are working to- 
wards, this is what DIAHLS is supporting... It needs the 
full support of the ISDMT, every member needs to really 
believe in the process, to understand it and be committed 
to it. They need to talk positively about it when talking 
to facility managers and other staff, they need to take 
roles in making it happen and really support it.” ISDMT 
meeting notes, 19th Jan 2011). 

The importance of such an approach is only affirmed 
by theory. “Leaders must foster learning and values... 
They need a sense of optimism that can help the system 
deal with complexity, risk taking, and uncertainty. They 
need to help the system maintain a coherent identity” 
[28]. Shared values and visions may, moreover, act as 
catalysts of change within a CAS, especially when they 
emerge through experience, providing the common en- 
ergy that encourages and enables commitment to action 
across people within a system [14]. 
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Second, in line with wider thinking [14], we have cre- 
ated spaces and processes where facility managers can 
be brought into relationship with each other, with col- 
leagues in the sub-district and with other local actors, to 
share knowledge and ideas, challenge each other and 
learn from each other. The community profiling initia- 
tive, for example, initially comprised a process of sharing 
ideas and experience in drawing onto physical maps. 
Facility managers commented on the value of seeing 
the world through others’ eyes, realizing also how 
knowledgeable community members are, and on having 
opportunities to talk with other local actors outside the 
pressurized atmosphere of their facilities. 

Several of the routine sub-district meetings have, fur- 
thermore, been adapted to provide opportunities to 
share and discuss experience about achievements, chal- 
lenges, and priorities and to give space to developing 
team working among facility managers and with sub- 
district colleagues responsible for human resource man- 
agement, supply management, and information systems. 
Meeting spaces also provide opportunities to develop 
new forms of accountability, to move away from the 
top-down approach perceived as checking progress to- 
wards targets and disciplining failure, towards a shared 
engagement about what enables and prevents progress, 
developing collective responses to tackling challenges: 
“it’s not about holding people accountable, but providing 
a space to be supportive in holding them accountable” 
(Researcher field notes, 6th June 2012; see also Key per- 
formance area process). 

Within DIAHLS, we have also thought quite carefully 
about meeting practices that allow more active engage- 
ment and ownership by all those present, rather than 
primarily being spaces where information or instructions 
are transmitted from managers to staff. Rotating the task 
of chairing, using rounds to allow each person to make 
an input to the meeting, and asking challenging ques- 
tions are, for example, ways of demonstrating equality, 
rather than reinforcing existing bureaucratic power bal- 
ances, and of sharing experience to identify where sup- 
port is needed. “The basic assumption is that opening up 
the meetings in these ways makes them less intimidating 
or threatening, and allows better communication — a 
two-way flow of ideas, between sub-district and facility 
managers in particular, but also to contextualize infor- 
mation about new activities for other programmes and 
support staff, and so encourage greater understanding 
and ownership of the activities” (Researcher field notes, 
6th June 2012). 

Third, through meetings and other routine activities 
the sub-district managers are also trying to instil a new 
culture of reflection and questioning — trying to encour- 
age facility managers to ask ‘does it make sense, how 
must it be done, can it be done better?’ As one manager 
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argued, “We have to change the way we do things, and 
that means not accepting how things currently are.” 
(Manager interview notes, 13th April 2012). 

The ‘KPA process; for example, specifically sought to 
respond to the negative perceptions around target set- 
ting by providing a space to allow facility managers to 
identify their own priorities whilst working within exist- 
ing planning frameworks and job descriptions. The in- 
tentions were to encourage understanding of higher 
level strategic priorities, forward planning at facility 
level, engagement with data relevant in setting priorities, 
and to develop “an attitude which looks at the under- 
lying causes of challenges so that you can actually get to 
the systemic issues you need to change to improve the 
whole picture” (Researcher field notes, 19th July 2012). 

Finally, sub-district managers are role modelling more 
systematic approaches to management through their 
personal practices [22]. For example, in how they con- 
duct staff appraisal discussions, being available on time 
for meetings and being respectful in their treatment of 
colleagues, as well as by coaching staff, running staff 
workshops in ‘difficult’ facilities and providing hands on 
support to weaker managers (Manager interview notes, 
ist April 2011; 15th October 2012; 7th December 2012). 
The research team members have, meanwhile, sought to 
role model reflective practice through their research ap- 
proach, for example, and in their approach to managing 
meetings. 

We also all recognize the power of language, through 
which managers are “able to articulate meanings, lend 
weight to collective action, and clarify the hoped for 
image of the organisation” [45]. The quarterly PDR meet- 
ing for local government facility managers, for example, 
has been deliberately renamed the Management and 
Communication meeting as a response to the sense that 
the managers “felt the name of the ‘PDR’ led people to 
worry, as they understood it to be essentially about criti- 
cizing them for not reaching their targets” (Researcher 
field notes, 6th June 2012). 

Ultimately, through these various practices, the sub- 
district managers are seeking “to use the intangible in 
combination with the tangible in ways that keep an eye 
on the goal [of PHC], but do things a bit differently — 
that’s bottom up power’ (Research meeting notes, 5th 
December 2012). 


Discussion 

These Mitchell’s Plain experiences show how, despite 
individuals’ agreement with overall policy goals, efforts 
to strengthen PHC confront facility managers’ collective 
weakness to engage in pro-active, local-level problem 
solving in support of population health and equity goals. 
This reluctance reflects collective mind-sets of passivity 
and risk avoidance rooted in three key experiences: 
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authoritarian cultures and histories; nearly two decades 
of centrally-driven policy and organisational change; and 
growing complexity in patient demands. These experi- 
ences, in turn, underpin vicious cycles of passivity, 
resistance to change, and further passivity, illuminating 
the ways in which, as Lipsky foresaw [29], sensemaking 
and the exercise of discretionary power are intertwined. 

The experiences also suggest that beyond developing 
guiding visions about PHC strengthening, leadership for 
PHC must support facility managers to take ownership 
of these visions collectively. The visions must make 
sense to them if they are to incorporate them into their 
practices and so exercise their discretionary power in 
pursuit of PHC goals. Lipsky [29] identified, for example, 
the importance of supporting front line workers through 
“ongoing processes of supportive criticism and inquiry. 
Built into every week of practice should be opportunities 
to review individual’s work, share criticisms, and seek a 
collective capacity to improve performance”. CAS theory, 
meanwhile, notes that, to support system change, leaders 
must create the conditions for the emergence of such 
change — in particular, by encouraging the cycles of ac- 
tion, feedback and learning that empower system actors 
to think and work differently [45]. 

Mitchell’s Plain sub-district managers have, partly in col- 
laboration with researchers within the DIAHLS project, 
initiated various activities intended to provide spaces of 
collective sensemaking to empower facility managers in 
these ways. Although these sensemaking efforts are still in 
their early days, the overall approach is affirmed by wider 
theory and empirical experience in two ways. First, in this 
broader literature, middle managers are recognized to play 
important and very particular roles as change inter- 
mediaries in organisational sensemaking, because they 
are positioned at the interface between an organisa- 
tion’s senior managers and front line employees [36]. 
From this position, “...interpretation is a key role. They 
need to ‘make sense’ upwards with senior managers, and 
laterally with peers and downwards with teams, to aid 
their interpretation of change intent and negotiate how 
change should be taken forwards” {25,33,46|. Second, to 
support organizational sensemaking, the literature sug- 
gests that middle managers must reflect on their own 
perspectives and behaviours, thinking about how they 
will help others through change. They will need to create 
spaces for conversation, reflection, and dialogue, perhaps 
around planning, resource allocation, and monitoring pro- 
cesses — allowing colleagues to revisit their own mental 
models, understand those of others, and develop shared 
interpretations and meanings of change [25,28,29]. More 
concretely, and directly reflecting Mitchell's Plain experi- 
ence, the ‘Leadership to support implementation of nurse- 
led community health programmes’ section below outlines 
the leadership practices applied within a UK experience of 
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organisational change around nurse-led community health 
programmes [47,48]. 


Leadership to support implementation of nurse-led 
community health programmes [Source: 47] 


¢ Created conditions for change through reflection, 
debate and challenge, workshops, skills audits, and 
education programmes. 

« Action learning to facilitate understanding of mental 
models and enable challenge to those models, leading 
to modification. 

¢ Worked with the ‘strange attractors’ (experiences or 
forces that attract the energies and commitment) that 
motivated practitioners (individual clients for some, 
whole communities for others). 

+ Introduced new relationships that led to new ideas, 
emergent behaviours, and work patterns. 

¢ Held multiple participatory events to encourage 
involvement, collaboration, and self-organisation. 

+ Established a few clear rules through discussion. 


Nonetheless, we recognize that the sensemaking activ- 
ities we have initiated will, inevitably, unfold in unpre- 
dictable ways over time. We do not expect that they will 
generate immediate and obvious change in sub-district 
performance, and instead we seek to encourage some 
change in meanings and practices that we hope will, over 
the longer-term, help embed a population health orien- 
tation within the local health system. Such leadership 
recognizes that “the order in organizational life comes 
just as much from the subtle, the small, the relational, 
the oral, the particular and the momentary as from the 
conspicuous, the large, the substantive, the written, the 
general and the sustained” (26). 

Some indications of the positive potential of our sense- 
making work lie in the perceptions of managers outside 
the sub-district. They report seeing a difference in the 
way things work in Mitchell’s Plain, in the willingness of 
staff to work together and tackle problems, and that they 
see the results in implementation of service delivery im- 
provements. Inside the sub-district, some facility man- 
agers also report finding the KPA process helpful as it 
provided a sort of on-the-job training and supported 
local level decision making to tackle problems. There are 
also clear indications of stronger peer support among fa- 
cility managers. 

However, our experience only confirms another theoret- 
ical insight — that it takes significant energy and attention 
to prevent personal and system practices falling back into 
pre-established patterns [26]. Maintaining energy around 
the local area groups that developed through the commu- 
nity profiling, for example, has been a challenge, and facil- 
ity managers have become dis-engaged from the activity 


HEALTH RESEARCH POLICY AND SYSTEMS 


Page 11 of 13 


(see Community profiling initiative above). Staff turnover 
also represents a significant challenge to institutionalizing 
new practices and meanings: for example, whilst the KPA 
process (see Key performance area process above) had 
some impact on some of those involved, newly appointed 
facility managers did not receive much orientation or sup- 
port for their engagement in this process and so were not 
really sure what they were doing. 

For middle managers, meanwhile, the Mitchell’s Plain 
experience indicates that the challenges of leadership 
include the ways in which their sensemaking efforts are 
filtered through other colleagues who may not adopt 
similar approaches or who may themselves be threat- 
ened by new approaches and ideas. In the face of facil- 
ity managers’ apparent passivity and lack of confidence, 
it may also be easy to slip back into micro-management; 
and, in the face of external demands, it is hard to maintain 
positive role modelling. To support organizational change 
middle managers not only need negotiation, persuasion 
and advocacy skills, but also, support from higher-level 
managers — such as the time and flexibility to sustain sen- 
semaking engagement with those on the front line of 
organizational change/policy implementation [22,36,46], 
consistent and positive messages about new activities from 
higher levels of the system, and a willingness to allow ex- 
perimentation to fine tune these activities. 


Conclusions 
Although exploratory, this analysis adds to the still limited 
body of work examining health system complexity [49], 
the influence of actors over policy implementation [50], 
and the leadership needed to support system-wide reforms 
in pursuit of population health and equity goals [12]. In- 
deed, we believe that this is the first paper specifically to 
begin to consider how sensemaking and discretionary 
power work together in challenging or supporting PHC 
re-orientation within a middle-income country health sys- 
tem, or to consider what a leadership of sensemaking for 
PHC entails. It has been made possible by the long-term 
and collaborative nature of the DIAHLS project, which 
supports the co-production of knowledge about the inner 
workings of the Mitchell’s Plain health system. Building 
on experience so far, we will continue to work together to 
strengthen, and track over time, our efforts to support 
PHC performance improvement. 

Our core argument is that: 


i. The system-wide population health re-orientation 
needed to sustain PHC in South Africa will only become 
a lived reality when the front line staff who work at the 
health system’s interface with the population bring it 
alive within their everyday routines and practices; 

ii, These agents’ sensemaking capabilities mean 
centrally-directed initiatives intended to strengthen 
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PHC are re-interpreted as implemented, with 
unexpected consequences that can include resistance 
to centrally-led activities (ie., exercising their 
discretionary power in ways that thwart such 
initiatives); 

iii. New forms of middle manager (and wider) 
leadership are required to nurture collective 
sensemaking around PHC goals and empower front 
line health staff to take ownership of these goals, 
and so exercise their discretionary power in their 
pursuit; 

iv. Mind-set changes, focused on concern for the 
population being served, the broader social 
determinants of health, and a willingness to act, are 
likely to be the fundamental basis for strengthening 
and sustaining PHC. 


In a complex adaptive system “...organisational change is 
not management induced. Instead, organizational change 
is emergent change laid down by choices made on the front 
line” {26]. Nurturing such change at the front line of the 
health system requires, therefore, new forms of leadership 
that enable sensemaking in support of change and unleash 
the collective power distributed across the system towards 
shared goals. 


Endnotes 

“The sub district population is around 510,000 and so 
is large in comparison to the WHO definition of a health 
district. In South Africa, the nine provinces each have 
constitutional authority for managing health services in 
their area and implementing health policy, with local 
governments having concurrent responsibility for man- 
aging aspects of primary health care. 

Since 1994, structural, management, and service de- 
livery developments within the Western Cape province 
health system have been guided by three inter-linked 
health policy documents: the 1995 Provincial Health 
Plan, the 2005 Comprehensive Service Plan (2005), and, 
most recently, the Healthcare 2030 policy document 
(available at http://www.westerncape.gov.za). Working 
within national policy frameworks, they provide a guid- 
ing vision for organizational change of a health system 
oriented towards population health needs and founded 
on a strong district health system. 
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Abstract 


Background: Using measles and tuberculosis as case examples, with a systems thinking approach, this study 
examines the human advice-seeking behavior of primary health care (PHO) physicians in a rural district of Pakistan. 
This study analyzes the degree to which the existing PHC system supports their access to human advice, and 
explores in what ways this system might be strengthened to better meet provider needs. 


Methods: The study was conducted in a rural district of Pakistan and, with a cross-sectional study design, it 
employed a range of research methods, namely extensive document review for mapping existing information 
systems, social network analysis of physicians’ advice-seeking practice, and key stakeholder interviews for an 
in-depth understanding of the experience of physicians. Illustrations were prepared for information flow mechanism, 
sociographs were generated for analyzing social networks, and content analysis of qualitative findings was carried 
out for in-depth interpretation of underlying meanings. 


Results: The findings of this study reveal that non-availability of competent supervisory staff, a focus on improving 
performance indicators rather than clinical guidance, and a lack of a functional referral system have collectively created 
an environment in which PHC physicians have developed their own strategies to overcome these constraints. They are 
well aware of the human expertise available within and outside the district. However, their advice-seeking behavior 
was dependent upon existence of informal social interaction with the senior specialists. Despite the limitations of the 
system, the physicians proactively used their professional linkages to seek advice and also to refer patients to the 
referral center based on their experience and the facilities that they trusted. 


Conclusions: The absence of functional referral systems, limited effective linkages between PHC and higher levels of 
care, and a focus on programmatic targets rather than clinical care have each contributed to the isolation of physicians 
and reactive information seeking behavior. The study findings underscore the need for a functional information system 
comprising context sensitive knowledge management and translation opportunities for physicians working in PHC 
centers. Such an information system needs to link people and resources in ways that transcend geography and 
discipline, and that builds on existing expertise, interpersonal relationships, and trust. 


Keywords: Health system, Measles, Pakistan, Primary health care, Social network analysis, Systems thinking, Tuberculosis 


* Correspondence: asmat.malik@uaconnect.edu.au 
"integrated Health Services, House 1-B, Street 50, Sector F-8/4, Islamabad, 
Pakistar 


Full list of author information is available at the end of the article 


; D , ~i 1+ Thic an fyonan A sce article dictribi ited tinder t terr of the Crea 
Malik et ai; licensee BioMed Central Lid. this is an Ope Access article distributed under the terms ie Creative 


” « MIA KY 
LY Vid G3 
voted Central Commons Attribution License (htto://creativecommons.org/licenses/by/2.0), which permits unrestricted use, aistnioution, and 
' " ss T (rr Man. P siDlic Domain 


reproduction in any medium, provides the original work 1S property credited, The Creative Comm 
httn://creativecommons.ora/publicdomain/zero/1.0/) applies to the data made available in this arti le 
iW //UrCagtivery J JIa/ 4b 3 } } , 


ata 
NCU 


MS, THINKING 918% HEALTH 


66 


Malik et al. Health Research Policy and Systems 2014, 12:43 
http://www.health-policy-systems.com/content/1 2/1/43 


Background 


Access to information is critical for creating and maintain- 
ing high performing primary health care (PHC) systems 
[1]. This aspect becomes especially important when front- 
line health workers (such as physicians, nurses, or allied 
health providers) face difficulties in diagnosing cases in 
PHC settings where consultations are of short duration 
[2], they are confronted with a wide range of medical 
problems [3], and their information needs are motivated 
by the specific needs of patient care [3,4]. Davies [5] de- 
scribes multiple sources of information that are available 
to physicians for assisting clinical decision making in 
difficult to diagnose cases, including clinical guidelines 
and research papers, as well as advice provided by other 
professionals [5] such as peers, fellow physicians, consul- 
tants, and teachers [6,7]. 

Huth et al. [8] note that physicians often seek advice 
from human sources when they are looking for readily 
available and convenient sources of information [8], 
mostly related to diagnostics, management, and referral 
strategies [7]. Text books, research papers, and other 
sources of information may not be adequate to answer 
their questions as many times physicians are also search- 
ing for support, guidance, affirmation, and feedback [9], 
which requires a synthesis of medical knowledge, patient 
information, and an understanding of the context of care 
[4], especially in complex cases [10]. 

The available studies provide some insights into how 
physicians seek information while working in PHC set- 
tings [2,5,11]. However, as this literature is largely con- 
fined to high-income countries, there is relatively little 
known about how physicians in low- and middle-income 
countries access or use information from human sources 
when faced with difficult to diagnose conditions. In 
these settings, where access to electronic information 
sources is often scarce, an understanding about advice- 
seeking behavior from human sources becomes particu- 
larly important. | 

Applying a systems thinking lens to understanding 
advice-seeking behavior in Pakistan’s PHC system is a 
key component of the healthcare delivery system [12], 
with PHC physicians being main actors within a com- 
plex health system [13]. Their behavior, linkages, rela- 
tionships, and interactions influence, and are influenced 
by, the system and its components [14]. Understanding 
and informing policy processes that are also influenced 
by human behaviors requires evidence that reflects the 
behavior of key actors, such as PHC physicians, and how 
these behaviors interact over time within social net- 
works. Despite efforts to maintain consistency and uni- 
formity in policy implementation through hierarchical 
control and command systems, there remain variations 
in how health professionals at the ‘street-level’ 
ment such guidance [15]. 
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Systems thinking encourages a dynamic and inter- 
related perspective of system structure and function, em- 
phasizing the importance of relationships between parts 
and whole, and the unpredictability of system behaviors 
[16,17]. Sterman [18] describes systems thinking as an 
“an iterative learning process in which we replace a re- 
ductionist, narrow, short-run, static view of the world 
with a holistic, broad, long-term, dynamic view, reinven- 
ting our policies and institutions accordingly”. A systems 
thinking lens allows us to recognize the importance of 
long-term change, the power of context, the role of 
guiding principles (rather than prescriptive control), the 
centrality of knowledge, and the enabling contributions 
made by interpersonal and inter-organizational relation- 
ships as vehicles for knowledge translation and exchange 
[19]. It is therefore a powerful lens through which 
advice-seeking behavior may be understood. 

Using methods grounded in systems science, this 
paper examines the human advice-seeking behavior of 
PHC physicians in a rural district of Pakistan in the pub- 
lic health sector, analyzing the degree to which the exist- 
ing PHC system supports their access to human advice 
and exploring in what ways this system might be 
strengthened to better meet provider needs. It goes be- 
yond an analysis of what information is sought by physi- 
cians — the common objective of needs-based studies — 
to understand how and from whom that information is 
sought. Using a systems lens, operationalized in part 
through social network analysis, explores the richness of 
interaction through both formal and non-formal rela- 
tionships in the context of the PHC system and their im- 
plications for clinical decision making. 


Research questions 
The specific research questions of this study are: 


e To what degree does the existing structure of the 
PHC system in Pakistan support physicians in 
accessing advice from human sources on difficult to 
diagnose cases? 

e To what degree are physicians satisfied with their 
current access to advice from human sources on 
difficult to diagnose cases? 

e What changes, if any, do physicians recommend to 
improve their access to advice from human sources 
on difficult to diagnose cases? 


Methods 

This study was conducted at the district level in Pakistan 
from January 2013 to August 2013. District Attock, pre- 
dominantly a rural district with a population of 1.6 million, 
was conveniently selected as a case illustration because of 
its proximity to the principal investigator [20]. 
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In order to align the study questions with the health 
problems that the district health department considered 
a priority, targeted key informant interviews were con- 
ducted with five district health administrators* and line- 
managers” of vertical health programs who were purpos- 
ively selected on the basis of their extensive knowledge 
of the information systems and their experience of wor- 
king in district health systems. Key informants were spe- 
cifically asked to nominate two priority health problems 
for use as case studies in order to map information flow 
mechanisms and analyze the advice-seeking behavior of 
physicians working in Basic Health Units (BHU). Tuber- 
culosis (TB) and measles were identified as key priority 
health problems to be used as case studies in this re- 
search. Despite nation-wide coverage of the National TB 
Control Program, TB remains a long standing problem 
across Pakistan, with key informants suggesting limited 
interest and action from BHU physicians related to TB 
program activities, especially the identification of new 
TB cases. Similarly, from January to April 2013, 192 
measles cases were reported in the Attock district, 
largely at secondary health care facilities [21], with just 
three cases identified by the BHU physicians despite the 
fact that most of these cases were from geographical 
areas where they should have been reported by their re- 
spective BHU. Based on recommendations from the key 
informants, the study aims were developed: 


e To document the flow of information® on diagnosis 
and management of TB and measles cases in the 
PHC system of Pakistan; 

e To describe the advice seeking behavior of 
physicians in situations with difficult to diagnose 
cases of TB and measles; 

e To explore physicians’ satisfaction with their access 
to advice in difficult to diagnose cases of TB and 
measles; 

e To identify and describe possible changes, if any, 
that physicians recommend to improve their access 
to advice in difficult to diagnose cases of TB and 
measles. 


In order to address these aims, with a cross-sectional 
study design and mixed-method approach, we employed 
three research methods comprising: i) mapping of exist- 
ing information systems; ii) social network analysis of 
physician advice seeking behavior; and iii) key stake- 
holder interviews for in-depth understanding of phy- 
sician experiences. 

Firstly, through documentary review (official memos for 
policy statements, job descriptions of doctors for roles and 
responsibilities, and training modules and guidelines for 
recordkeeping and reporting of suspected cases of TB and 
measles) and additional information obtained from five 
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key informants, we mapped the existing flow of informa- 
tion system for assisting physicians in diagnosing TB and 
measles cases. Illustrations of formal information dis- 
semination systems (Figures 1 and 2) were developed in 
the form of information flow charts showing the direction 
of flow of information and roles and responsibilities for 
providing information/feedback at various hierarchical 
levels. These illustrations were validated with the res- 
pective district health managers and BHU physicians for 
accuracy. 

Secondly, a semi-structured questionnaire was used to 
conduct a survey for mapping social and professional 
networks among BHU physicians. We adapted Blanchet 
and James methodological approach for mapping and 
analysis of social networks [22,23]. This approach com- 
prises three stages: i) defining the list of actors and 
members of the network; ii) analyzing the relationships 
between actors; and iii) analyzing the structure and dy- 
namics of social networks. 

The survey questions were structured to identify whom 
each BHU physician had contacted for advice whenever 
faced with difficult to diagnose cases of TB and/or mea- 
sles. All 49 physicians® were invited to participate through 
an official intimation from the Executive District Officer 
for Health. With one exception, all physicians participated 
in the survey. Participation was voluntary and each phy- 
sician completed his/her questionnaire in the presence of 
the principal investigator. In order to develop an egocen- 
tric network for analysis, each physician (ego) was asked 
to identify and name a person (alter) whom he/she had 
contacted to seek advice when faced with difficult to diag- 
nose cases of TB and/or measles [24,25]. As the scope of 
this research was limited to map the social networks from 
a BHU physician’s view only, alters were not contacted for 
confirmation. However, ‘name interpreter’ questions were 
asked of the physicians, designed to elicit further informa- 
tion about their respective alters, primarily covering their 
characteristics and relationship to the focal ego [25]. All 
different individual actors (egos and alters) were grouped 
based on their positioning (institutions/job titles) in the 
healthcare delivery system and placed under seven specific 
categories (Table 1). 

CINET software was used for generating directed- 
sociographs separately for TB and measles (Figures 3 
and 4). A single-headed arrow indicates a directed-tie, 
from an ego at the tail and respective alter at the arrow- 
head, indicating the direction of advice seeking [25]. The 
different shapes represent various categories of actors 
(node) whereas the line (tie) connecting between two 
nodes indicates the presence of a relationship for advice 
seeking. 

The analysis of sociographs indicated that a small 
number of physicians had sought advice when faced with 
a difficult to diagnose case of TB and measles (13 and 
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12 physicians, respectively). Given these findings, ana- 
lyses of network properties such as betweenness, cen- 
trality, distance, and reachability were not performed. 
The sociographs provided visual illustrations of the 
existing network for advice seeking. 

Thirdly, we conducted key stakeholder interviews in 
order to seek clarification and insights into the socio- 
graphs and to better understand the experiences of BHU 
physicians when they were faced with a difficult to diag- 
nose case of TB and measles. Based on the analysis of 
the findings from sociographs, the BHU physicians were 
divided into three groups: 


e Physicians who sought advice from a person 
designated by the district health department; 

e Physicians who sought advice from someone 
other than a designated person; 

e Physicians who did not seek advice from any 
‘other person. 


This grouping provided the basis for selecting 11 study 
respondents for in-depth interviews (sampled from each 
of the above categories). The key domains explored du- 
ring these interviews were reasons for seeking (or not 
seeking) advice, level of satisfaction under the present 
situation, and suggestions for improvement. 
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Interviews were digitally recorded and the average 
interview time was 20 minutes. All interviews were tran- 
scribed and content analysis was carried out by organi- 
zing coded data into categories, subthemes, and themes 
using an inductive process. 


Results 

Characteristics of BHU physicians 

The study population of 48 BHU physicians comprised 41 
males and 7 females. Average duration of government ser- 
vices was 6.7 years, ranging from 1 to 18 years. One-third 
(n = 16) of the study participants completed medical gra- 
duation overseas (mostly from Central Asian states). Fur- 
ther details are given in Table 2. 


The formal system of information flow for diagnosing TB 
and measles cases 
Under the National Tuberculosis Control Program (NTP), 
a District TB Coordinator (DTC) is responsible for ad- 
ministrative management of NTP activities at the district 
level. The DTC is also responsible for conducting trai- 
nings on Directly Observed Treatment Short-course 
(DOTS) strategy. 

The District and Tehsil Headquarters Hospitals 
(DHQH & THQH)* and Rural Health Centers‘ are estab- 
lished as TB diagnostic centers with nomination of a TB 
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| Figure 2 Illustration of flow of information (advice on Measles diagnosis) from district to BHU level under Expanded Program on 


focal person and upgradation of laboratory services for 
sputum and radiological examination service. 

All 61 BHUs across the district are grouped in 11 clus- 
ters and every cluster is attached with a TB diagnostic 
center on the basis of geographical proximity. This ad- 
ministrative change has provided a formal link between 
primary and secondary health care facilities but only for 
matters related to TB. In their monthly meetings, BHU 
physicians are required to meet with the TB focal person 
at their respective TB diagnostic facility, who is expected 
to provide follow-up of patients referred by BHU phy- 
sicians for confirmation of diagnosis, provide clinical 
advice and training, and advice on other programmatic 
issues. 

The flow of information under the Expanded Program 
on Immunization (EPI) follows the organizational hie- 
rarchy of the district health care delivery system which 
is aligned with the geographical boundaries of district 
and sub-district levels. In comparison to NTP, the BHUs 
are not clustered around another hospital or higher level 
health facility, but rather the administrative office of the 
Deputy District Officer for Health (DDOH). In order to 
seek advice on issues related to the EPI (including mea- 
sles), BHU physicians are expected to contact their re- 
spective DDOH (their immediate supervisor at Tehsil 
level). Further, in contrast to NTP, they do not have di- 
rect linkage with secondary care hospitals. The DDOH 


is also responsible for training on EPI, with facilitation 
from the District Officer for Health. 


Structure of professional and social networks for 

advice seeking 

The structure of the social network for advice seeking 
reveals that 27% BHU physicians (n= 13) contacted a 
human information source when faced with a difficult to 
diagnose case of TB (Figure 3). 

Ideally, as per the specified criteria established by the 
NTP [26], all were expected to contact the TB Focal 
Persons at their respective TB diagnostic centers (Figure 1); 
however, only two physicians contacted their designated 
source for advice. Among those that sought advice from 
an alternative source, four directly approached the DTC 
for an expert opinion, two consulted each other, whereas 
one sought advice from the respective DDOH. All con- 
tacted someone within the district except three who pre- 
ferred to seek advice from a tertiary care hospital located 
externally to the district (Figure 3). The sociogram de- 
monstrates a mixed pattern of advice seeking which is not 
consistent with expectations under the NTP. 

The structure of the social network for advice seeking 
for measles demonstrates that 25% of BHU physicians 
(n= 12) contacted a human information source when 
faced with a difficult to diagnose case of measles 
(Figure 4). 
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Table 1 Grouping of actors and institutions in seven 
categories 


No Category _Number 
1 District Coordinator for National TB Control | Program | 1 

2 Deputy District Health Officers 5 

3 TB Diagnostic Health Facilities 1] 

4 BHU Physicians 49 

5 Consultants in Secondary Care Hospitals 1 

6 Consultants in Tertiary Care Hospitals 3 

7 Private Practitioners 1 


As per the departmental directions, all of the BHU 
physicians were expected to contact their respective 
DDHOs; however, only three physicians contacted their 
immediate supervisor. This finding was observed in one 
Tehsil only, with no physicians in the remaining five 
Tehsils seeking advice from their respective DDHOs. 
Four physicians preferred to contact a pediatrician in a 
single THQH. A similar number of physicians sought 
advice from another BHU physician. Only one physician 
contacted a private practitioner for advice. This pattern 
shows some differences from the advice-seeking network 
for TB because none of the physicians sought advice 
from outside the district. 
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The next section presents thematic analysis of the 
qualitative research findings from in-depth interviews. 


Qualitative findings from the in-depth interviews 

The transcripts were coded and then organized into 
categories from which three subthemes and one main 
theme were identified. The analysis process is given in 
Table 3. The data from qualitative research findings are 
presented starting with the subthemes and their relation 
to the categories and concludes with how they contri- 
bute to the main theme. 


Subtheme 1: lack of confidence in available resources 

All physicians had been working in BHUs for at least a 
year and were aware of the constraints and challenges 
associated with the BHU environment. They lacked con- 
fidence in the available resources for clinical guidance 
within the system. 


Physician considers first line officer incompetent 

Among those physicians who did not seek advice in rela- 
tion to a complicated case of measles or TB, some sug- 
gested that the available experts within the district 
health care delivery system were not of the relevant field 
or lacked competence. 
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“First of all, the doctors posted at TB diagnostic center 
should be competent enough. The doctor posted 

! presently over there is not competent; I have no faith 

: in him. Moreover, the position of medical specialist at 
THQH should be filled. Presently, a doctor is available 
at this hospital who has done his diploma in 
cardiology. Personally speaking, I don’t think he is 
competent enough to deal with TB patients.” 


Another physician reflected: 


“He is not an experienced person. Sometimes, he shows 
an X-ray to me to seek guidance on what to do further. 
For example, recently, he showed me an X-ray for 
advice whether that patient should be put on 


Table 2 Characteristics of BHU physicians in Attock 
district he 
Characteristics 


Proportion 
(Number) 

~ 66% (n= 32) 
50% (n = 24) 
54% (n = 26) 

66% (n = 32) 

85% (n=41) 


Attained medical education within Pakistan _ 
Employed as regular government employees 
Established private clinic besides government job 
Attended formal training on EPI* before joining service 


Attended formal training on TB DOTS*™ before 
joining service 


*EPI, Expanded Program on Immunization; **TB DOTS, TB Directly Observed 
Treatment Short-course. 
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| Figure 4 Network of advice-seeking for difficult to diagnose cases of measles among BHU physicians. 


Streptomycin. I don’t consult him for advice because 
he is not that competent.” 


Physicians were aware of the limitations of their 
reporting officers or immediate supervisors and reported 
seeking advice from them in administrative issues only. 
Four physicians felt that they had not encountered any 
difficulty in diagnosing any difficulty and hence, did not 
seek advice from anyone. 


Lack of support system for clinical guidance 

For clinical advice related to measles, physicians re- 
ported trust in the opinion of pediatricians, while for 
TB, physicians reported willingness to contact the dis- 
trict TB coordinator. The need to improve access to 
those with relevant expertise was expressed, as was a 
formal mechanism within the PHC system for facili- 
tating access to clinical guidance in a timely fashion. 
In addition, BHU physicians also reported being over- 
worked with organizing different campaigns and acti- 
vities and were left with little time for patient care: 


“There should be a mechanism through which we can 
meet the specialists or have training session with them 
at least after every 2 to 3 months so that they know 
the BHU doctors and understand their problems. 
Although BHU doctors are performing their duties, it 
is a general perception that nothing is done at the 
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Table 3 Analysis process moving from categories to themes: ee 
CA gs CATEGORIES = 
1. Specialist reachable over phone for advice 

if personally known 


CATEGORIES 
1. Physician considers first line officer 
incompetent 


2. Lack of support system for clinical 
guidance 


3. Lack of access to latest literature/research 
from the Health Department 


4. Lack of functional referral system 
SUBTHEME 1 


Lack of confidence in available resources 


2. Focus of departmental meetings on 
targets and not clinical guidance 


SUBTHEME 2 
Unpredictable support 
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CATEGORIES 
oy, Lack of trust in organizational pathways for 
seeking information for patient care 


2. Personal professional/social network 
(locally and distant) for patient care 


SUBTHEME 3 


Reliance on personal (rather than organizational) 
resources to safeguard patients’ interests 


MAIN THEME 


Formal organizational structures, including supervisory support and technical guidelines, not adequate 


level of BHUs. There is no one in a BHU after 11 am. 
These days, BHU doctors are heavily engaged in 
activities like Dengue control, measles vaccination 
campaign, and other official meetings. There is a single 
doctor posted at BHU level and when he is to remain 
away on account of all these official engagements, who 
will take care of the patients?” 


Lack of access to latest literature/research from health 
department 

The main reliance of physicians for information was 
through other doctors and text-books: 


“If I require advice then I would prefer to reach out to 
my seniors because they have practical experience. 
I have never read research papers.” 


The internet was used to access information on com- 
plicated cases by some physicians; however, access to the 
internet was not available at every BHU. Other resource 
materials identified were pamphlets and brochures from 
representatives of pharmaceutical companies and hand- 
books on clinical guidelines. The health department did 
not provide such information: 


“There should be some system that BHU doctors 
remain updated with new developments. We will keep 
on making our personal efforts but there should be an 
official system, which should facilitate our access to 
updated information and research.” 


Lack of a functional referral system 

There was also a lack of a functional referral system. As 
such, even when physicians consulted someone or re- 
ferred them to another provider within the system, there 
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was no mechanism in place to ensure feedback to the re- 
ferring physician: 


“In case of diagnosis, I make a provisional diagnosis 
based on my clinical knowledge and books and then 
refer the patient accordingly. If the patients come back 
to my center afterwards, I get feedback from them.” 


Reiterating the non-functional feedback system, the 
physicians considered that it was more convenient to 
simply refer a suspected case of TB to a TB diagnostic 
facility without differentiating between a difficult or easy 
to diagnose case. 


Subtheme 2: unpredictable support 

Interviewed physicians suggested that the health system 
worked in ‘spurts, with the fluctuating functionality of 
the system physicians were driven to look for alternate 
sources when faced with complicated cases requiring 
advice. 


Specialist reachable over phone for advice, if personally 
known 

Specialists at times were available and reachable over the 
phone for advice on management/complications. This, 
however, was true in cases only when they personally 
knew the specialists and were close enough to them to 
call them. 


“I often contact Dr. A when I face problem in 
managing the treatment of TB patients. As far as 
history, symptoms and signs of TB are concerned, 
! think, I don’t have any problem with that. 
Further, 20-30% diagnosis is confirmed by the 
laboratory tests at the diagnostic center. 

Dr. A is our master trainer for TB program and I 
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Know him — that is why I often contact him for further 
advice.” | 


Moreover, the referral center had the facilities for car- 
rying out further tests. 


“The patient presented with enlarged and matted 
cervical lymph nodes. I talked to Dr. B about her and 
then she was referred to him because we thought she 
might require a biopsy. They have more facilities 
available at the THQH level.” 


Focus of departmental meetings on targets and not clinical 
guidance 

Within the health department, most meetings focused 
on setting targets and achieving them. Clinical manage- 
ment was discussed only when a senior doctor was 
personally interested in clinical management and also 
competent to carry out such discussions. The support 
system to provide clinical guidance was non-existent. 
Thus, the system fluctuated as different doctors were 
appointed: 


“We participate in so many monthly-meetings because 
we have to remain in touch with our superiors and 
seniors. These meetings are not focused on diagnosis 
and patient management, rather the stress is on 
achieving our targets irrespective of whether these are 
achievable or not. Focus is more towards paperwork 
but not towards practical work.” 


Subtheme 3: reliance on personal (rather than 
organizational) resources to safeguard patients’ interests 
Physicians were motivated to seek advice and provide 
care to their patients. They realized the constraints 
within which they worked and had to provide best care 
to the patients. 


Lack of trust in organizational pathways for seeking 
information for patient care 

The physicians knew that there was no formal mecha- 
nism to seek advice within the system and there was a 
lack of trust in organizational information-seeking path- 
ways as well. This was based on their own experience of 
not finding competent physicians posted at the higher 
health care facilities. Through their own professional 
network they sought advice: 


“Dr. C is a friend of mine and I can contact him 
easily.” 


One physician described how a personal experience 
with illness influenced his advice-seeking behavior and 
the need to safeguard patient's interest to seek care: 
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“Because | had suffered from the same situation 
myself; therefore, I directly contacted Dr. B. | had 
become so afraid of TB at that time because I was in 
my final medical year and exams were due in just a 
few months and my case was so much mismanaged. It 
is generally said that one becomes more sympathetic to 
a patient who shares the same experiences of suffering 
that a health provider has also gone through. That is 
why I referred this patient to a proper place so that 
they do not wander from place to place and get a 
trustworthy opinion.” 


The referring physicians indicated that there were in- 
ternal contradictions in communicable disease reporting 
mechanisms: the reporting of cases of vaccine-preventable 
diseases, including measles, was received with some am- 
bivalence by the higher authorities, given that they had 
been reporting high vaccine coverage for many years. The 
consequences for communicable disease surveillance were 
understandable, but undesirable; physicians were inclined 
not to document and report such cases. In complex cases 
of measles, it was easier to avoid negative consequences 
for the referring physician by not using formal reporting 
channels, but instead referring them directly to a pe- 
diatrician, outside the PHC system: 


“The more important thing is that every case of 
measles should be reported so that proper information 
is available in the health system. However, this is not 
the case in practice. Once you are discouraged by the 
high-ups that such cases are not to be reported, the 
doctor himself avoids reporting cases of measles so that 
he does not face any consequences. Being positioned at 
BHU level, we do not like to bear the responsibility. 
Therefore, if any case of measles turns up with or 
without complication, we do entertain him and refer 
him to a higher level of care but without the official 
reporting that is required in such a case.” 


Personal professional/social network (local and distant) for 

patient care 

Many physicians used personal links to seek information 
for their patients. On encountering a complicated case, 
physicians reported reliance on other physicians/special- 
ists they knew personally and could call over the phone 
to consult (even when specialists were present within 
the district health care system): 


“It depends upon the specialty or the type of the cases. 
For example, for Gynecology and Pediatrics I contact 
Dr. X (Gynecologist) whereas for pediatrics I contact 
Dr. Y (Pediatrician). For other cases I contact my 
colleagues and friends even outside the district because 
every specialist within the district cannot be contacted 
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on personal relationship. For example, it ts not 
possible to contact Dr. Z (Medical Specialist at 
DHQH) because he is not known to me.” 


Main theme: formal organizational structures, including 
supervisory support and technical guidelines, not 
adequate 

The main theme identified from the perspectives of inter- 
viewees illustrates how their work setting and informa- 
tion/advice-seeking patterns are driven by the constraints 
in organizational structures. It is difficult for BHU phy- 
sicians to ensure comprehensive responsibility to their 
population if their efforts are not supported by specialized 
services available at secondary and tertiary levels of health- 
care [1]. Getting their part of the system right does not 
help if other system components do not provide the re- 
quired support [27]. Non-availability of competent super- 
visory staff, a focus on improving performance indicators 
rather than clinical guidance, and the lack of a functional 
referral system, collectively create an environment that is 
non-conducive for improving patient care. In the long 
term, physicians develop their own strategies to overcome 
these constraints. Their advice-seeking patterns largely de- 
pend on access to information systems and their contacts 
with colleagues within and outside the primary health care 
system. Non-responsiveness of the healthcare delivery sys- 
tem creates an environment where they work in isolation. 
Ultimately, it has negative consequences both for shoul- 
dering responsibilities and improving quality of care. 


Discussion 
As noted by the WHO, the building blocks of health Sys- 
tems (service delivery; health workforce; information; 
medical products, vaccines and technologies; financing; 
and leadership and governance (stewardship)) while in- 
dependently critical for system effectiveness, do not op- 
erate in isolation: complex relationships between the 
building blocks may help (or hinder) the overall ability 
of a system to use resources for improving health [12]. 
Human resources and information flow (two of the six 
building blocks), therefore, play critical roles in connect- 
ing sub-systems, promoting ongoing learning, and driv- 
ing performance improvement. Examining how those 
within health systems share, access, and apply informa- 
tion is therefore important for better understanding how 
existing system structures and functions support or hin- 
der learning and improvement. 

Here, we discuss the implications of this study’s fin- 
dings across four domains: System organizing, system 
networks, system dynamics, and system knowledge [13]. 


System organizing 


The designers of the NTP and EPI have achieved the 
program-specific institutional arrangements that they 
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require at the sub-district level by clustering BHUs around 
TB diagnostic facilities and offices of the DDHOs, respec- 
tively, with additional administrative and managerial sup- 
port provided from the district level. These institutional 
arrangements have provided an organizational skeleton 
for administration, training, and reporting through setting 
rules and regulations and assigning roles and responsibil- 
ities. However, these efforts have focused on establishing 
managerial control over program activities rather than 
nurturing a learning oriented environment that BHU phy- 
sicians could rely on for advice, particularly when con- 
fronted by difficult to diagnose patient presentations. 
Consistent with previous reports from other jurisdictions 
[2,20,28], this study has identified non-availability of ex- 
pertise and lack of a functional referral system as key defi- 
ciencies within the exiting PHC system, exacerbated by 
limited interconnections between BHU physicians and 
those providing higher, specialist levels of care. Moreover, 
even where designated centers do exist (such as in TB 
diagnostic facilities), BHU physicians rarely perceive these 
as authoritative or expert learning resources. Similarly, 
EPI line-managers (DDHOs) were infrequently contacted 
for advice, primarily due to a perceived lack of clinical 
knowledge related to the management of measles. The ab- 
sence of a functional and reliable referral system further 
compounded this situation. Consequently, BHU _physi- 
cians were more reliant upon their own knowledge and 
relationship to identify appropriate sources for advice. 


Systems networks 

As noted, the findings from this study demonstrate that 
the information-seeking behavior of BHU physicians are 
diverse, driven by both the context and their network of 
available relationships, and do not follow formal orga- 
nizational structures. The boundaries of information- 
seeking behavior do not appear to be limited by geography 
(district/province), type of health sector (public/private), 
or levels of healthcare (primary, secondary, and tertiary). 
Results from the qualitative investigation suggest that the 
presence of appropriate social ties (relationships) largely 
determine how physicians in this setting seek information 
from other human sources. In the Pakistani health system 
(as in other systems), specialist doctors are posted and 
available in the secondary and tertiary care hospitals with 
few formal linkages to the PHC infrastructure. Our study 
findings show that, while BHU physicians were aware of 
the human expertise available within and outside the 
district, their information-seeking behavior was largely in- 
fluenced by their informal interactions and relationships 
with senior specialists. 

Health systems are driven by humans and their interac- 
tions [29]. However, the existing PHC system in Pakistan 
does not appear to support or nurture relationships bet- 
ween junior and senior physicians. Participants in this 
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study suggested a number of ways in which such connec- 
tions may be fostered, such as greater in-service training 
events, more regular opportunities to work beside senior 
doctors, or even activities that build stronger connections 
between public and private sectors. 


System dynamics 

Health systems are not static and adjust and readjust 
over time as contexts change, feedback is provided, and 
histories develop. The human instrument, unlike other re- 
sources, has the ability to cope and adjust to its environ- 
mental needs. Lack of response from higher authorities, 
discouraging attitudes, especially in the case of reporting 
vaccine preventable diseases (e.g., measles), and absence 
of feedback mechanisms, may demoralize frontline health- 
care providers leading them to develop their own stra- 
tegies outside formal accountability systems. Despite the 
limitations of the existing PHC system, a number of phy- 
sicians in this study reported proactively using their pro- 
fessional linkages, experiences, and relationships to seek 
information and refer patients. However, there currently 
exists no process whereby referred patients may be 
followed-up in PHC centers. As a result, many physicians 
were not confident of the outcomes of referral under these 
circumstances, with patients often ‘lost in the system’. Al- 
though patient referral was occurring, long-serving public 
sector physicians were skeptical that a functional referral 
system could or would be created. 


Systems knowledge 

While authority lies in knowledge [30], health system 
managers in this study were largely viewed as adminis- 
trators lacking in clinical expertise. As a result, BHU 
physicians’ information-seeking behavior tended to be 
driven by perceptions of clinical expertise rather than 
hierarchical positioning or seniority. With inadequate 
use of resources to support physician training in TB 
DOTS and EPI prior to joining services, and a focus on 
management targets rather than clinical outcomes, BHU 
physicians developed their own learning and improve- 
ment strategies. Yet, without a supportive learning cul- 
ture, many opportunities for better managing, sharing, 
and improving knowledge were missed, leading to isola- 
tion for many BHU physicians. In the absence of formal 
systems that meet their needs for clinical advice, indi- 
vidual physicians draw on their own networks of re- 
sources, and this in turn creates emergent organization 
that partly compensates for the gaps, but without ad- 
dressing their causes. 


Limitations of the study 

This study has a number of limitations. Firstly, the scope 
of the investigation was restricted to two specific dis- 
eases, TB and measles, in the context of a single district. 
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Therefore, while the results are informative, they cannot 
be generalized to other diseases and geographical areas. 
Secondly, the network data were based upon the infor- 
mation provided by the PHC physicians. We did not 
contact the respective senior doctors for confirmation of 
the reported ties because it was beyond the scope of this 
research. Thirdly, this study has been conducted in pub- 
lic sector PHC facilities only and does not cover private 
sector PHC general practitioners who are a major source 
of service provision in Pakistan. However, as more than 
half of BHU physicians also work in the private sector, it 
is possible that findings may be similar across practice 
settings. Despite these limitations, the lessons learned 
could potentially be used in designing studies for provid- 
ing comparative analyses in different contexts. 


Conclusions 

Through the examples of TB and measles, this study has 
demonstrated how and why PHC physicians seek informa- 
tion when confronted with difficult to diagnose cases, and 
the challenges of creating learning systems that support 
continuous improvement. Given the number and diversity 
of patient presentations seen each day by PHC physicians, 
it is possible that there exists a more generalized need for 
high quality, reliable, and accessible information. Yet the 
absence of functional referral systems, limited effective 
linkages between PHC and higher levels of care, and a 
focus on programmatic targets rather than clinical care, 
have each contributed to the isolation of physicians and 
reactive information-seeking behavior. The advice-seeking 
behavior observed in this study may be explained by phy- 
sician’s lack of confidence in available information re- 
sources, an unpredictable patient referral system, and a 
greater belief in personal rather than organizational re- 
sources for ensuring high quality patient care. Through 
interpreting these findings in partnership with those re- 
sponsible for system design, it may be possible to assist 
provincial health departments in Pakistan to review the 
modalities of providing support to PHC physicians, espe- 
cially in the country’s post-devolution phase. At this time, 
organizational and structural changes have a high poten- 
tial of being entertained and implemented. The study fin- 
dings underscore the need for a functional information 
system comprising context-sensitive knowledge manage- 
ment and translation opportunities for physicians working 
in PHC centers. Such an information system needs to link 
people and resources in ways that transcend geography 
and discipline, and that builds on existing expertise, inter- 
personal relationships, and trust. 


Endnotes 
*Fxecutive District Officer for Health, District Officer 
for Health and Deputy District Officer for Health. 
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District Coordinator for National Tuberculosis Control 
Program, District Superintendent for Vaccination, Pro- 
gram Director for District Health Development Center, 
Coordinator for District Health Information System. 

“For this paper, the flow of information is defined as 
formal mechanism of information exchange (reporting 
on case detection of TB and measles, through trainings, 
and advice seeking) in public sector health care delivery 


system at district level. 
‘Out of 61 BHUs in district Attock, physicians were 
appointed in 49 BHUs and the remaining positions were 


vacant at the time of survey. 

“Secondary Care Health Facilities. 

‘A higher level PHC facility at a level between BHUs 
and secondary care health facilities. 
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Abstract 


Background: Many full-time Ugandan government health providers take on additional jobs — a phenomenon called 
dual practice. We describe the complex patterns that characterize the evolution of dual practice in Uganda, and the 
local management practices that emerged in response, in five government facilities. An in-depth understanding of dual 
practice can contribute to policy discussions on improving public sector performance. 


Methods: A multiple case study design with embedded units of analysis was supplemented by interviews with policy 
stakeholders and a review of historical and policy documents. Five facility case studies captured the perspective of 
doctors, nurses, and health managers through semi-structured in-depth interviews. A causal loop diagram illustrated 
interactions and feedback between old and new actors, as well as emerging roles and relationships. 


Results: The causal loop diagram illustrated how feedback related to dual practice policy developed in Uganda. As 
opportunities for dual practice grew and the public health system declined over time, government providers 
increasingly coped through dual practice. Over time, government restrictions to dual practice triggered policy 
resistance and protest from government providers. Resulting feedback contributed to compromising the supply of 
government providers and, potentially, of service delivery outcomes. Informal government policies and restrictions 
replaced the formal restrictions identified in the early phases. In some instances, government health managers, 
particularly those in hospitals, developed their own practices to cope with dual practice and to maintain public 
sector performance. Management practices varied according to the health manager's attitude towards dual practice and 
personal experience with dual practice. These practices were distinct in hospitals. Hospitals faced challenges managing 
internal dual practice opportunities, such as those created by externally-funded research projects based within the 
hospital. Private wings’ inefficiencies and strict fee schedule made them undesirable work locations for providers. 


Conclusions: Dual practice prevails because public and private sector incentives, non-financial and financial, are 
complementary. Local management practices for dual practice have not been previously documented and provide 
learning opportunities to inform policy discussions. Understanding how dual practice evolves and how it is managed 
locally is essential for health workforce policy, planning, and performance discussions in Uganda and similar settings. 


introduction Indonesia and Bangladesh, held second jobs [1-5]. In 
Dual practice, when health workers employed full time some contexts, dual practice can be broader than private 
by the government take on additional jobs, is widespread _ for-profit sector service delivery — including both research 
in developing countries, particularly those with growing and NGO work. Researchers and policy-makers in devel- 
private sectors. Recent studies found that 29% of physicians oping countries display increasing interest in how dual 
in Cote d'Ivoire, 35% of physicians in Vietnam, 42% in _ practice affects the health system [6,7]. 
Sri Lanka and 41% in Zimbabwe, , and as high as 80% in Uganda is one of these countries. The country has a 
vibrant private health sector. Within this, the private 
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period of civil war and remains underfunded to date. 
Uganda's private for-profit sector is large, fragmented, and 
disorganized, yet very little is known about it. Although 
there are growing discussions about public-private part- 
nerships in health [8], dual practice seldom features on 
these agendas and data on this topic is scarce. In Uganda, 
in 2005, a nationally representative survey of private 
health facilities found that more than half (54%) of private 
sector doctors also declared being formally employed in 
the public sector [9]. While estimates from public facilities 
do not exist, in general, health providers and_policy- 
makers perceive that almost all government-employed 
health workers have dual practice. In addition, dual prac- 
tice has been rising in importance on the policy agenda 
due to media reports of adverse health service delivery 
outcomes [10,11], as well as suspected linkages to absen- 
teeism and the wastage linked to it [12,13]. A recent study 
aiming to establish policy-makers’ research priorities re- 
vealed that a principal concern was dual practice that 
was “reported to greatly affect the performance of the 
public sector. The dual [practice] of public health 
workers has implications on quality and management of 
health care delivery such as indiscipline, time loss and 
poor work ethics” [7]. 

Despite these concerns, data on dual practice in 
Uganda and elsewhere is scarce. Although many types of 
health providers are believed to engage in dual practice, 
the available literature examines dual practice rather 
narrowly, generally only from the perspective of physicians. 
Furthermore, existing studies provide few answers to 
questions related to the policy and management of dual 
practice, beyond agreement that the effects of dual prac- 
tice on the organization of the health system and service 
delivery can be either positive or negative, and that these 
effects, and related policy responses, are highly dependent 
on the local context [14-16]. For example, if well managed, 
dual practice may help prevent doctors from leaving the 
country by enabling them to supplement salaries without 
adversely affecting stock of doctors in the country. Con- 
versely, if poorly managed, absenteeism and pilfering may 
negatively affect public sector standards of care and con- 
tribute to inefficiencies. The factors and interactions that 
drive these effects have not been explored extensively. 
Presumably, these factors depend on how dual practice 
has evolved and how it is managed in a particular health 
system. 

Studying the dynamic aspects of dual practice and re- 
lated interactions both within and outside the boundaries 
of a health system requires a departure from the linear, 
theoretical models found in the literature [2,17-20]. A 
more appropriate model acknowledges the holistic, com- 
plex, and adaptive nature of health systems and their 
broader environment. Complex systems are composed of 
many interacting components that organize themselves in 
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dynamic ways, are unpredictable in the long-term, and are 
able to learn from past experiences [21-23]. A research 
design acknowledging complex systems’ features, as well 
as potential interactions due to contextual factors, is ideal 
to guide the exploration of phenomena, such as dual 
practice, from multiple perspectives. It facilitates the 
exploration of complex system characteristics, such as 
feedback, emergence, and self-organization. In this paper, 
we explore how dual practice evolved in the Ugandan 
health system and how it is currently managed, in an 
urban environment — the city of Kampala, with an active 
private sector. Additionally, using systems approaches, we 
attempt to reflect on why dual practice persists and the 
factors underlying its current management. Understanding 
dual practice holistically in the Ugandan context provides 
a basis for exploring potential policy options. Gaining an 
in-depth understanding of the role of dual practice at vari- 
ous levels of the system can help policy-makers and health 
managers to strengthen management of dual practice and 
its consequences. 


Methods 

Research design 

We use case studies of urban public health facilities to 
investigate the role of dual practice and the key patterns 
and interactions that it motivated in the health system. 
Review of policy documents, as well as qualitative inter- 
views of policy stakeholders were used to supplement a 
qualitative survey of workers and their managers in the 
study facilities. In addition, during the data analysis phase, 
we developed a causal loop diagram to illustrate key fac- 
tors and related feedback influencing dual practice in the 
current context. This paper presents only a sub-set of data 
that were collected as part of a sequential, exploratory 
mixed methods study. 

We purposefully selected five public sector health facil- 
ities in Kampala, Uganda, to represent the various levels 
in the Ugandan government health system: two Health 
Center III facilities, one Health Center IV facility, and 
two urban hospitals (see Table 1 for case characteristics). 
Health Center III facilities have a general outpatient clinic 
and a maternity ward. Health Center IV is a larger facility 
than the Health Center III facilities, with the capacity for 
inpatient services and some emergency operations. Re- 
gional referral hospitals have specialized clinics, and are 
staffed by a variety of cadres, including medical specialists. 
The national referral hospital also has research and teach- 
ing components, in addition to medical service provision. 
Within these case studies, individual respondents were 
purposefully selected to ensure that, at each facility, the 
perspectives of providers (doctors and nurses) and the 
facility manager, were captured [24]. At each facility, the 
sample included the health facility manager (in-charge in 
health centers, directors or department heads at hospitals), 
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Table 1 Summary of selected cases 


Facility type Case A ‘ Case B CaseC CaseD CaseE 
Health Center |II* X xX 

Health Center IV Xx 

Hospital X X 
Location 

Central X X X 
Periphery X X 

Staff composition 

General practitioners X X xX 
Specialists X X 
Nurses X X X X X 
Filled positions 121% 74% 90% 144% 


90% 
Source: Ministry of Health - Human resources for health audit DSL « . oe 
*Note: Health Center Ill units are supposed to be staffed by Clinical Officers 
and Nurses - although sometimes units do have a Medical officer as well. 


as well as a doctor and nurse recommended by the health 
facility manager. Within the larger hospital, the sampling 
occurred at the level of the clinical specialty [1], and there- 
fore included multiple manager-level respondents, as well 
as a nurse and a doctor recommended by each of them. 
Within the smaller hospital, the sample included the dir- 
ector and two providers recommended by them. 

Policy stakeholders included purposefully selected indi- 
viduals from professional councils, relevant government 
ministries, private not-for-profit medical bureaus, private 
sector hospital administration, and the local district health 
office. The main criteria guiding the sampling was the 
extent to which a stakeholder would know policies on 
dual practice either at the national level or within their 
organization, would have a stake in the development of a 
policy on dual practice, and present a unique perspective 
on dual practice in the Ugandan context. 


Data collection instruments and field work 

A review of policy documents was undertaken before the 
data collection, and as these documents became available. 
The main areas of interest were the existence and content 
of policies, actors, and events that played a role in the evo- 
lution of dual practice in Uganda, as far back in time as 
possible. 

The interview guides contained questions about the evo- 
lution of dual practice in Uganda, providers’ motivation to 
engage in dual practice, advantages and disadvantages or 
challenges linked to dual practice, facility-level policies 
and management approaches, and potential policy recom- 
mendations. Interviews with policy stakeholders focused 
on policy-related questions, as well as on the evolution of 
dual practice in the health system. Data collection took 
place during July-August 2012. The interviews were 
conducted in English. Interviews were recorded, unless 
respondents preferred otherwise. 
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Data analysis 

All of the recordings were transcribed and stored in Atlas. 
ti v. 7. A preliminary, exploratory coding structure was 
constructed based on initial readings of the transcripts 
and on the conceptual framework derived from a systems 
approach to health markets and theories related to sys- 
tems thinking and health worker motivation [21-23,26,27]. 
Multiple rounds of coding focused on refining the scheme 
[28]. During coding and analysis, memos were devel- 
oped to capture changes in the coding structure, as well 
as emerging reflections. 

Text query results from Atlas.ti were arranged in matri- 
ces for within and cross-case analyses, according to the 
methods suggested by Miles and Huberman [24]. For each 
case, matrices were developed by theme (e.g., informal 
organizational policies), with focus on the embedded units 
of analysis (e.g., summarizing and contrasting the per- 
spectives of health facility managers, doctors, and nurses). 
Cross-case theme analyses focused on exploring the differ- 
ences and similarities between the five cases, specifically 
by health facility type. The policy stakeholder interviews 
were analyzed for emerging themes, along the same lines 
as the case studies. References to the analysis and any 
quotes are labeled according to respondent type, to main- 
tain anonymity. 


Causal loop diagram development 

Although it was not an explicit goal of this study, inter- 
views with policy-makers revealed that the role of dual 
practice and the government policy on dual practice chan- 
ged over time, and that examining this progression might 
be useful for understanding the current policy situation. 
Based on discussions with policy-makers and case study 
respondents, as well as available historical accounts, we 
developed a causal loop diagram (CLD). The CLD illus- 
trates the events, actors, and interactions — or the under- 
lying mental model and system behavior — that fostered 
the emergence of dual practice policy responses over time 
in the Ugandan health system, facilitating the visualization 
of complex system patterns and characteristics, such as 
policy resistance, feedback, and adaptation [29,30]. 

The CLD was developed using Vensim PLE Plus [31]. 
It was challenging to recreate the history of dual prac- 
tice, particularly in the distant past. An account of the 
medical profession in East Africa, which included de- 
tails about the emergence and development of dual 
practice and the private sector from the perspective of 
physicians, helped to identify relevant early events 
from the 1960s and the 1970s [32]. Recent events have 
been identified from the in-depth interviews conducted 
for this study and available documents. The CLD was 
refined through various iterations, to ensure that the 
relationships, interactions, and direction of feedback 
were most plausible. 
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The CLD uses standard notation: “a positive (+) arrow 
from variable A to variable B means that A adds to B, or, 
a change in A causes a change in B in the same direction; 
a negative (—) arrow from A to B means that A subtracts 
from B, or, a change in A causes a change in B in the 
opposite direction” [33]. Some of the relationships create 
feedback loops. These loops are reinforcing if the vari- 
ables influence each other in the same direction. Loops 
are balancing if they influence each other in different 
directions. The thickness of the line denotes researcher's 
emphasis on a relationship, for illustrative purposes. 
Dashed arrows highlight key, probable relationships 
identified through this study. The question mark (?) indi- 
cates an unknown relationship. This is not the recom- 
mended notation, as it is preferred to make explicit the 
“multiple causal pathways connecting the two variables” 
[33]. However, data currently do not exist to sufficiently 
tease out how dual practice affects service delivery. For 
example, while we know that dual practice can affect 
systems positively and negatively, whether and how much 
dual practice contributes to adverse service delivery out- 
comes is unknown. 


Ethical approvals 

Ethical approvals were obtained from the Institutional 
Review Board of the Johns Hopkins Bloomberg School 
of Public Health (IRB No. 4371), the Makerere University 
College of Health Sciences — School of Public Health 
Higher Degrees, Research, and Ethics Committee (IRB 
No. 11353), the Mulago Research Ethics Committee 
(Protocol no. 249), and the Uganda National Council 
for Science and Technology (Ref. No. SS 2883). 


Results 

Twenty-three interviews with doctors, nurses, and health 
managers from various types of facilities, as well as 13 
policy-stakeholder interviews were conducted. None of 
the respondents approached for an interview declined to 
speak to us, although a few preferred that our interview 
not be recorded. Respondent characteristics are displayed 
in Table 2. About half of our health facility respondents 
reported having dual practice at the time of the interview, 
or having been previously been involved in private sector 
work. 

The CLDs that follow display the factors associated 
with the presence of dual practice in the system and the 
emergence of current management practices and policies. 
They illustrate three phases to describe the emergence of 
dual practice in Uganda: pre-independence through the 
1960s, 1970s through the 1980s, and the 1990's through 
the present. Table 3 complements the CLDs by illustrating 
a timeline of critical events that affected the policy and 
management of dual practice. 
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The remainder of this section first describes, in each 
phase, the feedback and interactions that emerged in rela- 
tion to government policies on dual practice, as well as, 
more broadly, the development of a mixed health system 
in Uganda. It concludes by describing how dual practice is 
currently managed in the government facilities included 


in this study. 


Phase 1: Dual practice policy before Uganda’s 
independence and through the 1960s 

Figure 1 illustrates a relatively simple system, showing 
no feedback or unintended consequences, where a nascent 
private sector does not initially provide sufficient incen- 
tives for providers to engage in dual practice. 

During this time, government restrictions on dual prac- 
tice are formal — written and enforced: dual practice is 
allowed only after government hours. Some dual practice 
opportunities exist, however, demand is low due to high 
satisfaction with the government benefits. Few government 
providers chose to engage in dual practice, generally 
seeking the autonomy provided by private practice. No 
evidence was found that dual practice raised concerns 
about adverse health service delivery outcomes. Even as re- 
strictions on dual practice became stronger after Uganda's 
independence, the Ugandan government was able to pro- 
vide government health workers with sufficient financial 
and non-financial incentives (e.g., satisfactory wages and 
the prestige of working in a government institution, re- 
spectively). One of the policy-stakeholder respondents 
confirms the general sentiment in this period [32]: 


“The assumption was, that what the government pays 
can cater for what you require in real life. [...] in the 
60%, a medical officer, medical assistant, the nurse, 
was capable of catering for everything they required, 
the basics of life [with the government salary alone]. 
And they were held with high esteem, they were very 
ethical. | mean a medical officer would walk with his 
head high.” ~ Ministry of Health policy stakeholder 


Phase 2: Dual practice policy in the 1970s and 1980s 
Figure 2, illustrates a second phase, during which the 
Ugandan system undergoes instability of military rule and, 
eventually, civil war. 

During this period, instability affects the health workforce 
in multiple ways: through reduced infrastructure and 
government budgets, as well as through persecution of 
health providers for political reasons. These country-level 
hardships are intensified by the broader global recession. 
The multiple crises cripple the government health system, 
and mark the beginning of several decades of low govern- 
ment salaries. While the job security and prestige related 
to government service are still important, the government 
financial incentives are no longer sufficient for providers 
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Table 2 Interview respondent characteristics 


Facility-based respondents 
Case A 
Gender Male 0 
Female 3 
Yrs. in service <10 1 
10-19 0 
20-29 2 
30+ ) 
Profession Nurse 2 
General practitioner 0 
Clinical officer 1 
Specialist 0 
Dual practice Yes 
TOTAL 
Policy stakeholders 
Gender Male 
Female 
Sector Public/government 


Professional associations 

Private for-profit 

Private not-for-profit 
TOTAL 


‘Years in service not available for one of the respondents at this facility. 


who remain in the system. Many government providers 
resign at this time, or leave the country. Increasingly, 
government providers who remain in the system seek 
additional income through dual practice. The same policy 
stakeholder explained: 


“(With] the economic downturn of the 70s, then the wars 
that have been associated with [Amin and Obote’s] 
regime, the salary did not have any meaning anymore. 
[...] The global economy has changed, impacting 
everyone, |...| the country, with all the hardships it’s 
had — the economy has not been able to cope with the 
many social needs. That's why salaries across all public 
servants have remained very low and therefore public 
servants have to look for alternative survival 
mechanisms.” — Ministry of Health policy stakeholder 


As the public sector increasingly suffers and govern- 
ment providers “look for alternative survival mechanisms’, 
this period leads to the first large-scale development of 
the private sector, after Asians (including doctors) were 
expelled from Uganda for political reasons, many of the 
Ugandan government doctors who remained in the country 
re-opened the former Asian private practices and many of 
them were perceived to have dual practice. At this time, the 
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Case B Case C ‘CaseD' 7" Case E Nr. (%) 
1 1 2 4 8 (35%) 
2 2 1 7 15 (65%) 
1 1 0 0 3 (13%) 
1 1 0 5 7 (30%) 
0 1 1 2 6 (26%) 
] 0 2 3 6 (26%) 
2 2 1 2 9 (39%) 
1 1 0 0 2 (9%) 

0 0 0 1 (4%) 
0 0 2 9 11 (48%) 
10 (43%) 
23 

12 (92%) 
1 (8%) 
5 (38%) 
4 (31%) 
3 (23%) 
1 (8%) 
13 


increasing concerns about the quality of services provided 
in private for-profit medical practices (while not proven to 
be linked to adverse health and health system outcomes), 
contributes to suspicion around dual practice, specifically 
related to potential damage to the quality of services in the 
public sector and pilfering of government medical supplies. 
Consequently, the government begins imposing strong, 
formal restrictions on dual practice, a strict ban on dual 
practice and, at one point in 1972, closes all private clinics. 
As shown in Figure 3, these strong, formal restrictions to 
dual practice trigger provider protests and resignation, 
and contribute to provider migration, both of which com- 
promise the supply of government health providers. Inter- 
national sanctions on the military government and a 
declining economy made salaries of civil savants unattract- 
ive. Increasing protests and advocacy from professional 
associations eventually lead to the government relaxing 
restrictions. Weaker restrictions, which allowed dual prac- 
tice after government hours, reduced the threat to govern- 
ment providers and diminished the undesirable feedback. 
During the 1980s, the global debt crisis and the sub- 
sequent structural adjustment program fuelled the 
development of the private sector, while, at the same 
time, constraining government budgets [34]. In this con- 
text, the financial benefits of working in private practice 
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Table 3 A timeline of critical events and government policy on dual practice 


Year Event 

Pre- ‘Nr. of African health professionals 
growing 

1962 Ugandan independence 

Post-1962 Govt. suspicions about private 


sector growing 


Transition to military rule and 
civil war 


1972 Asian doctors expulsed 


Ugandan doctors take over 
private practices 


1974 Government shuts down 
private practices 


Provider protest advocacy to 
allow dual practice 


Late 1970's Broadly, international sanctions 
on military government led to 
economic collapse and decline 
in government salaries relative 


to cost of living 


Government changes policy on 
dual practice as incentive for 
govt. providers 


1980's Govt. suspicions about dual 
F practice and private sector 

Paes strengthen 

2000's Rapid private sector growth, 


especially after system recovered 
from civil war, creates increasing 


Dual practice policy ¥ 


practice allowed after govt. hours 


Strong formal govt. restrictions: dual 
practice not allowed 


Weak formal govt. restrictions: dual 
practice allowed after govt. hours 


Weak, formal govt. restrictions: dual 
practice not allowed 


No formal govt. restrictions 


Informal govt. restrictions on dual 
practice, with weak influence 


~ Weak formal govt. restrictions: dual 
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Consequences 


None 


No immediate effects 


After 1970's events, restrictions 

to dual practice contributed to 
resignations from government 

services and provider migration — 
therefore reducing the number 

of govt. providers 


Dual practice is a coping 
mechanism for providers 
remaining in Uganda 


nr. of dual practice opportunities 


2005-2007 MOH tests ban on dual practice 
in few hospitals 
2009-2010 Office of President establishes 


Medicines and Health Service 
Delivery Monitoring Unit 


Increasing nr. of policy discussions 
around dual practice, absenteeism, 
ghost workers 


Increasing concerns about the 
contribution of dual practice 
to decreases in quality and 
access to care in both public 
and private sectors 


Providers threaten to resign 


Dual practice important coping 
mechanism 


Providers threaten to resign in 
response to discussions of ban 


significantly exceeded those of the public sector, and moti- 
vated government providers to engage in dual practice. 
Government restrictions remain formal, but weak, at this 
time. Based on the available information, we propose that 
during this eventful and tumultuous period, dual practice 
and the incentives related to practicing in the private 
sector complement incentives for government service. 
Moreover, restrictions on dual practice without any fur- 
ther measures to address the government health system 
contribute to a decrease in the number of government 
providers. The dashed lines in Figure 2 highlight these 
proposed influences. 
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Phase 3: Dual practice policy from the 1990s to the 
present 

Figure 4 illustrates the changes in the system from the 
1990s to the present: the private sector grows significantly 
as Uganda recovers from civil war and privatization is en- 
couraged through the structural adjustment program [34] 
and well-financed vertical health projects and clinical re- 
search initiatives. 

Dual practice opportunities grow quickly in a context 
of rapid private sector growth, as well as of increasing 
donor-funded research and NGO projects, generally housed 
within public facilities. Due to an ever constrained budget 
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, and growing population demand, the government health 
system cannot offer providers an alternative to dual 
practice. Private practice during this period promises 
significant financial incentives, particularly in contrast 
| to low government salaries, but lacks the job security 


_ Figure 2 Causal loop diagram illustrating factors influencing dual practice during the 1970's and 1980's. The causal loop illustrates the 
second period of interest: the health system is challenged by broader contextual events — such as the civil war and the global debt crisis. As 
demand for dual practice grows, so do opportunities for government providers. The government becomes increasingly suspicious of potential 
adverse effects and, at first, imposes a ban on dual practice. A balancing loop first results in unintended negative consequences (See Figure 3 for 


further details). 
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Adverse health and health 
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sector * 
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Figure 1 Causal loop diagram illustrating factors influencing dual practice from pre-independence through the 1960's. The causal loop 


illustrates the first period of interest: a simple system with little demand for dual practice. It is important to highlight that no feedback loops were 
identified in this phase. 


and prestige that are still associated with government 
practice. The increasing population demand, as well as the 
significant earnings possible through private practice, make 
dual practice a frequent coping mechanism for government 
providers. In the absence of formal rules to manage dual 
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Provider 
protest/resignation 


Govt. restrictions to - | Nr. govt. providers 
dual practice in dual practice 
oe 
Professional 
associations 
advocacy 
+ 
Adverse health and health 
system outcomes in public 
sector 
Figure 3 Focusing on the restrictions loop. Starting with the 1970's, strong restrictions to dual practice trigger unintended consequences 
through a balancing feedback loop — a decrease in the number of government providers. Subsequently, successful advocacy efforts to ease 
restrictions eventually dampen their effects on the broader health workforce, although restrictions remain in place they are acceptable to the 


provider population. This figure re-draws the CLD diagram to better illustrate the factors influencing these unintended consequences. 


J 


practice, health facility managers develop their own formal 
and informal practices for mitigating detrimental effects 
of dual practice, such as absenteeism, while retaining the 
government health workforce, despite low salaries and 
poor infrastructure. 

The absence of a formal policy on dual practice was 
confirmed by interview participants and also by our re- 
view of Ministry of Health and Ministry of Public Service 
policy documents. As our respondents illustrate below, 
current government restrictions are informal — unwritten, 
not enforced, and based on expectations of provider be- 
havior in the public sector. 


“I don’t think there is a clear policy saying that there 
is no dual practice {...| we are expected NOT to do it 
[...]. [Health workers] know what's supposed to be the 
normal, but are kind of forced to do it, as I’ve said, to 
improve a bit on their earnings. |...| We don’t come 
out to fight it. | can’t tell someone please don’t go the 
other end, because there's a reason that is pulling this 
person to go, and I have no control over that. All I can 
do is to make sure enforcing that this person is here 
with me at the right time, for 7 or 8 hours. So we can't 
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influence what happens beyond that {...] I cannot 
influence the earnings. [...| The person has the needs, 
and I can’t satisfy the needs in any other way {...] I 
cant provide alternatives.” — Government official 1 


“A lot of policies are implemented while they are just 
known by the policy-makers but they are not written 
down. So, we know about dual practice and the policy 
is that |...| it should be left as it is. That people can be 
allowed to do dual practice. |...] It is not written. It’s 
not written at all, but they should not take too much 
of public time to do it. [...] Unfortunately, there is no 
mechanism to enforce how much public time people 
are going to take because |...| a lot of things that have 
gone wrong, including this dual practice, have gone 
wrong because of poor regulatory systems.” — 
Government official 2 


Interviews with policy stakeholders revealed that the 
government initiated periodic attempts to formalize 
government restrictions on dual practice, motivated by 
suspicion around dual practice due to media coverage 
of adverse health outcomes and poor public sector 
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Figure 4 Causal loop diagram illustrating factors influencing dual practice during the 1990’s to the present. Dual practice opportunities 
grow exponentially, as it becomes more attractive to government providers working in an underfunded and over-burdened public system. A 
formal, written government policy does not exist. Local facility-level coping mechanisms emerge to mitigate negative consequences of dual 


practice on the health system. Periodic threats for increasing restrictions re-activate the feedback loop presented in Figure 3. 


performance. Additionally, restriction attempts are trig- 
gered in the context of budget discussions, media reports 
about ghost workers, and increased concerns about qual- 
ity of care in both public and private sectors as indicated 
by adverse health service delivery outcomes linked to 
absenteeism, pilfering of drugs, and patient deaths in the 
private sector from suspected malpractice. 

Escalating policy discussions around formalizing re- 
strictions on dual practice are often met with provider 
protests, triggering the feedback displayed in Figure 3 
and the government goes back to “keeping quiet”, in this 
case meaning informal restrictions. A couple of the policy 
stakeholders provides examples of such events, which also 
illustrate that the government increasingly recognizes the 
role of dual practice in the system, particularly in the 
absence of changing government pay. 


~2005-—2007: “The [high level official] gave a directive 
that it should stop. |...| He said: ‘Officer, we are going 
to work out the methodology of implementing it |...] 
But we shall not do it broadly across the country, we 
shall test it in some hospitals.’ So we came {to one of 
the hospitals], and communicated what the {high level 
official] had done, and said, these people |at this 
hospital] said: ‘We hear you loud and clear, but let's 
agree if I cannot take that prescription, am I free to 
leave the government job? So that I can go to the other 
side {meaning private practice]? [...] we either stay or 
go? Is that what you're trying to communicate to us?’ 
We said: ‘Yes’. Within two days |...| the [hospital] 


director came rushing to the headquarters to say: 
‘Guys, stop talking about dual practice because 
everybody is winding up to go.’ So, the [government 
official] went back and told the |high level official]: 
‘We tried to test it in {a hospital] and all the 
consultants are not bothered - they want to leave 
[this hospital].’ - and dual practice has gone on.” — 
Ministry of Health policy stakeholder 1 


In response to the cycles of uncertainty related to in- 
formal government restrictions to dual practice, as well 
as to coping with potential negative consequences of dual 
practice on public sector performance, we found that 
local, facility-level and department-level management 
practices can develop in government facilities. Facility 
management practices arise in response to increasing 
number of government providers with dual practice and 
aim to reduce any adverse service delivery outcomes in 
the public sector. As long as dual practice remains an 
incentive for providers to remain in government service 
(and sufficient resources to incentivize providers other- 
wise do not exist), these facility management practices 
could potentially weaken any policy restrictions to dual 
practice and any related negative effects on the govern- 
ment workforce. 


Local management practices for dual practice 

Data from the case studies revealed that, in the absence 
of a formal, written policy on dual practice, health man- 
agers develop their own approaches to coping with and 
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managing dual practice on a daily basis. Table 4 sum- 
marizes the approaches identified through this study. 
These facility-level management practices encourage the 
presence and performance of their staff during govern- 
ment hours although these codes of practice are generally 
unwritten. For example, no respondents described dual 
practice being addressed directly during regular staff 
meetings. Instead, respondents described informal, one- 
on-one consultations: health managers intervening with 
providers in private, often in response to an issue related 
to provider performance. 

In one case, a health manager fostered a culture of 
flexible scheduling, i.e., all senior doctors get one day or 
certain afternoons to dedicate to their other activities, 
whether research or dual practice, in exchange for reporting 
to duty on other days. According to the unit’s manager: 


“We tried to create a bit of flexibility and say, ok, all 
of us must be on station in the morning, and let’s take 
turns to cover the evening. And may be trying to bring 
the evening time a bit forward to, to allow people to 
earn some extra earning. |...| When I see the outputs, 
then I don’t complain. Yes, and sometimes they come 
and start early, before 8 oclock and if someone is here 
by 7 and even comes back on the weekend to clear if 
there is any backlog, I think, really, I can only say 
thank you because I can’t pay them more than they 
earn.” — Case E, Health manager 


This particular arrangement was not only facilitated by 
the fact that the unit manager was understanding of the 
reasons why providers would engage in dual practice 
and had an output-oriented supervision style, but also 
by the fact that the majority of doctors working in this 
unit worked in the same private health facility, which 
was close to their government location. 

Within the larger facilities, formal policies included, 
for example, having a private wing, where doctors and 
nurses could see and get paid for private patients under 
the auspices of the government facility, or limiting nurses’ 
night duties so as to deter them from taking up full-time 
dual practice during the day. 

Most of the health managers interviewed had a generally 
favorable attitude towards dual practice, not discouraging 
it within their facility. Their attitude stemmed from their 
own personal experiences where, in the past, they also had 
no choice but to take on additional jobs to compensate for 
government sector shortcomings. It also stemmed from 
broader frustration at not being able to enforce attendance 
policies and not having the necessary tools to adequately 
monitor health workers (an exception was the health 
manager for Case C, who expressed high confidence in 
public accountability). The principal tool available to 
managers for holding health workers accountable were 
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attendance registers, which could be easily falsified. In 
this context, dual practice was generally tolerated 
within government facilities. Health managers empha- 
sized the need to prioritize the completion of govern- 
ment duties and, to the extent possible, tried to 
introduce incentives for improved performance in the 
public sector. 


“I don’t stop anybody from doing that. What |...] I tell 
them is that: priority is a core job and your core job is 
the public service. Once you do my work well, then I 
don't mind about what you do next.” — Case D, 
Nurse/Health Manager 


These management approaches, generally lenient with 
respect to dual practice, seemed to mitigate providers’ 
exit from the government health workforce. Additionally, 
they also seemed to tackle broader issues of provider per- 
formance, such as absenteeism. In Case B, providers 
reported being able to manage their two jobs without con- 
flict. One of the providers reported seeing dual practice as 
a privilege: “if you want to reward yourself by doing an 
extra job, you have to make sure we [in the government 
sector] are covered” (Case B, doctor). Nevertheless, because 
of the broader health system issues, where managers lack 
tools to properly enforce policies in general, approaches to 
manage dual practice also had shortfalls. For example, in 
Case C, the in-charge reported that providers who were 
found with multiple jobs (often caught in the private fa- 
cilities), were asked to quit them in favor of government 
service. While this manager reported confidence in this 
approach, the other respondents from the facility reported 
that almost everyone in the facility engaged in dual prac- 
tice, but this was not discussed with the manager. Some of 
these approaches also created tensions among staff. The 
flexible scheduling mentioned earlier was not available to 
non-physicians and therefore friction arose from time to 
time among work teams. The private wing is one of the 
dual practice policy interventions listed in the literature 
[15], however, in the study context, it was perceived to be 
inefficient, and the infrastructure only marginally better 
than the rest of the facility. 


Discussion 

This paper is one of the few contributing empirical evi- 
dence on dual practice policy and management prac- 
tices in Uganda and low- and middle-income countries, 
more broadly. It illustrates how dual practice policies 
changed over time in the Ugandan system and how this 
phenomenon is currently managed within a sample of 
government facilities. It also attempts to use the existing 
data to reflect on and to explain why dual practice persists 
and the current approaches that have developed in the 
study context. 
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Due to a series of health and non-health sector events, 
feedback, and learning, dual practice has become an 
informal, yet integral component of a government health 
workers’ incentive package. This package has also evolved 
over time to one where job security and prestige remain 
important, but no longer sufficient due to some of the 
lowest salaries in the region, poor system infrastructure, 
and increasing patient loads. In a situation where the 
government cannot offer financial or non-financial al- 
ternatives to substitute dual practice (i.e., improve the 
incentives for sole public practice), the official policy for 
dual practice shifted from formal restrictions to one 
based on informal expectations. Any attempts to formalize 
restrictions is met with unintended consequences due to 
policy resistance and emerging feedback, threatening the 
stability of government health workforce more broadly. 

Our data confirmed the existence of self-organization 
through local, facility-level management practice, which 
allow health managers and providers to cope with working 
in both the public and private sectors. The purpose of 
these practices was not necessarily to curb dual practice, 
but to maintain performance of the public sector by 
ensuring the presence of providers and, at the same 
time, to achieve an optimum balance between government 
workers’ public and private activities and needs. Some 
of these management practices were easy to identify and 
describe, e.g., the ones guided by a health manager, as in 
the example of the hospital department. Other manage- 
ment practices, based on individual negotiations, presum- 
ably depended on internal provider networks, whose 
development, and also decline, could not be captured 
through our study methods. Most frequently, health 
managers found opportunities to intervene as common 
symptoms of dual practice that threaten public sector 
performance, such as absenteeism, triggered concerns. 
These management practices could potentially minimize 
destabilizing effects occasionally arising from the policy 
feedback and resistance. 

Our exploration revealed two issues that are relevant 
beyond the issue of dual practice policy and management 
and perhaps also beyond the Ugandan context. First of all, 
public sector performance management emerged as an 
area with significant shortfalls. In the absence of tools and 
support for rewarding good performance and punishing 
poor performance, the tacit, indirect approach to man- 
aging dual practice does not sufficiently empower health 
managers to supervise and enforce boundaries for govern- 
ment employees, who must fulfill their duties in both the 
public and private sectors. Also, because the nature of 
dual practice differs among nurses, general practitioners, 
and specialists, cadre-specific management approaches 
and tools might be appropriate. 

Second, the nature of the Ugandan health system, and 
that of many sub-Saharan African countries, is very 
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different than it was immediately after independence. 
Initially designed around the public sector, the private 
sector and particularly the private for-profit components, 
have been treated with suspicion and not integrated 
within a broader vision for the health system. Presently, 
the pluralist health systems that dominate low- and 
middle-income countries cannot be ignored. As a ma- 
jority of the population, including the poor, relies on the 
private for-profit sector, increasing government stew- 
ardship is necessary to maintain the highest standards 
of service delivery [35]. In this context, providers engaged 
in dual practice could serve as a channel for reaching the 
private for-profit sector and the synergies between govern- 
ment practice and private for-profit practice must be 
recognized. 

How dual practice actually affects the health system 
and service delivery remains one of the key unknowns. 
While the literature and study respondents acknowledged 
both positive and negative effects of dual practice, in 
most low- and middle-income contexts, Uganda included, 
actual effects on the health system are unknown. Further- 
more, issues such as shortfalls in quality of care, absentee- 
ism, and efficiency gaps in public spending have broader 
root causes and can only partly be attributed to dual prac- 
tice. A better understanding of the dual practice effects on 
providers, health facilities, and the broad health system 
would help governments to better calibrate their policy 
approach and to explore options for reaching a better 
balance between public and private sector spheres in 
health care. 


Strengths and limitations 

This study represents one of the few exploring dual 
practice holistically, from multiple perspectives (doctors, 
nurses, managers, policy stakeholders), and by applying 
systems thinking tools, such as the CLD. Only a few 
examples of CLDs exist in health research [36-39]. The 
researchers established credibility and confirmability of 
the findings by triangulating the data from the interviews 
across multiple types of providers, and, where possible, 
through the document and policy review. 

The conclusions are constrained by several limitations. 
Much of the early history of dual practice in Uganda relies 
on a single source and it was not possible to verify the 
events or written government documents we mention. Be- 
cause the case studies were based in a large urban center, 
generalizations to rural Uganda, where the opportunities 
for private practice are substantially different, are not 
possible. We could not explore dual practice in private 
not-for-profit or for-profit facilities or include additional 
cadres believed to engage in dual practice (e.g., clinical 
officers). The large hospital was much more complex than 
the other cases included in our study, and perhaps deserved 
to be studied in greater depth. Although the information 
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presented in the CLD was triangulated across all available 
data sources, it could not be validated with study respon- 
dents as it was developed after the data collection ended. 

Future research into how dual practice is managed 
by public facilities and how private for-profit facilities 
incentivize and contract with their providers, would be 
helpful. More in-depth studies looking at dual practice 
from the perspective of other cadres, such as clinical 
officers, or of rural practitioners could provide additional 
insights into this phenomenon. The facility-level manage- 
ment mechanisms described here could inspire formal 
policies aimed at minimizing the negative consequences 
of dual practice, while helping to seize opportunities for 
public-private sector synergies. The effects of dual practice 
on service delivery outcomes, such as quality of services 
and access to care, have not been established in the litera- 
ture, although there is consensus that dual practice likely 
contributes both positively and negatively. Validating the 
CLD and translating it into a system dynamics model 
could be relevant in policy discussions as a platform for 
testing various policy scenarios and anticipating unin- 
tended feedback in the system. 


Policy relevance 

The unintended feedback revealed through the CLD, at 
times detrimental to the public health sector, confirms the 
recommendations of previous studies, which proposed 
that a ban on dual practice would not be practical or 
effective [14-16,40]. Periodic threats of banning dual 
practice also risk destabilizing the public health sector 
in places like Uganda, primarily by reducing the supply 
of government health workers. The private for-profit 
sector allows government providers the additional finan- 
cial resources that the Ugandan government is currently 
not able to supply. In a relationship of mutual depend- 
ency, government providers in dual practice allow for the 
growth of the private for-profit sector in the context of 
limited health workforce and increasing population 
demand. 

In the short term, the Ugandan government should 
consider the promotion of policies that are flexible to 
local adaptions to promote access and quality of services 
in the public sector, while at the same time allowing suf- 
ficient income for government providers. Informal, local 
adaptations to managing dual practice exist in Uganda 
and provide a natural experiment for various dual prac- 
tice policies. In the long term, the Ugandan government 
should consider broader improvements to public sector 
management and increasing the resources available to 
the health sector, as well as increasing synergies with the 
private sector. 

Reforms currently under discussion in Uganda include 
health insurance and performance-based contracts, both 
would change how providers are paid. Such reforms could 
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potentially provide an entry point for strengthening public 
sector management in general, and therefore provide 
health facility managers the tools they are currently 
lacking to manage dual practice. As dual practice is un- 
likely to disappear in the short term, its existence and 
role in the health sector cannot be ignored during the 
design and implementation of major health reforms in 
Uganda and other countries where dual practice exists. 
Potential unintended effects (feedback) should be an- 


ticipated based on past events related to dual practice 
and dealt with accordingly. 
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Abstract 


Background: Of the three million newborns that die each year, Uganda ranks fifth highest in neonatal mortality 
rates, with 43,000 neonatal deaths each year. Despite child survival and safe motherhood programmes towards 
reducing child mortality, insufficient attention has been given to this critical first month of life. There is urgent need to 
innovatively employ alternative solutions that take into account the intricate complexities of neonatal health and the 
health systems. In this paper, we set out to empirically contribute to understanding the causes of the stagnating 
neonatal mortality by applying a systems thinking approach to explore the dynamics arising from the neonatal health 
complexity and non-linearity and its interplay with health systems factors, using Uganda as a case study. 


Methods: Literature reviews and interviews were conducted in two divisions of Kampala district with high neonatal 
mortality rates with mothers at antenatal clinics and at home, village health workers, community leaders, healthcare 
decision and policy makers, and frontline health workers from both public and private health facilities. Data analysis 
and brainstorming sessions were used to develop causal loop diagrams (CLDs) depicting the causes of neonatal 
mortality, which were validated by local and international stakeholders. 


Results: We developed two CLDs for demand and supply side issues, depicting the range of factors associated with 
neonatal mortality such as maternal health, level of awareness of maternal and newborn health, and availability and 
quality of health services, among others. Further, the reinforcing and balancing feedback loops that resulted from this 
complexity were also examined. The potential high leverage points include special gender considerations to ensure 
that girls receive essential education, thereby increasing maternal literacy rates, improved socioeconomic status 
enabling mothers to keep healthy and utilise health services, improved supervision, and internal audits at the health 
facilities as well as addressing the gaps in resources (human, logistics, and drugs). 


Conclusions: Synthesis of theoretical concepts through CLDs facilitated our understanding and interpretation of the 
interactions and feedback loops that contributed to the stagnant neonatal mortality rates in Uganda, which is the first 
step towards discussing and exploring the potential strategies and their likely impact. 


Keywords: Causal loop diagram, Child health, Health systems research, Methods, Neonatal mortality, Systems 
thinking, Uganda 
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Background 

With around three million babies dying each year within 
their first four weeks of life (neonatal period), virtually 
all (99%) occur in low- and middle-income countries 
(LMICs) [1]. Moreover, the most recent progress reports 
on global trends in neonatal mortality have shown alarm- 
ingly slow progress, if any, in curbing mortality rates 
among neonates, the slowest being in sub-Saharan Africa 
[1]. Three quarters of these neonatal deaths occur within 
the first week of life and at least 1 million die on the first 
day of life [1]. Uganda is one of the high burden countries 
in sub-Saharan Africa where the rate of decline in neo- 
natal mortality has remained below the global average 
over the past 20 years, with an estimated 28 newborn 
deaths per 1,000 live births (a total of 43,000 deaths per 
year) in 2011 [1]. While child survival programs have 
tended to focus on pneumonia, diarrhoea, malaria, and 
vaccine-preventable diseases, all of these contributed to 
death after the first month of life. There is no documented 
progress in targeted approaches to prevent death around 
birth and the first week of life (Figure 1). 

Several studies have tried to understand the reasons 
for this slow progress, employing a range of methods in- 
cluding logistic regression models [2-4], cross tabulations 
[5], principal component analysis, and simultaneous mul- 
tiple regression models [6]. The vast majority of these 
studies focused on only one or a few aspects of the prob- 
lem in isolation. Most notable were lifesaving interven- 
tions or key strategies geared towards improving access to, 
or coverage of, health services. Interventions that do not 
employ the holistic approach often focus on the symptoms 
neglecting the underlying root causes, thereby resulting in 
the reoccurrence of the problem. However, neonatal mor- 
tality is a very complex problem involving multiple factors 
and actors and requiring multiple interrelated and simul- 
taneous strategies to be able to effectively address it, 
including the difficult challenge of changing attitudes, 
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perceptions, behaviour, and practices [7]. This complexity 
calls for employing more holistic approaches that acknow- 
ledge the complexity of neonatal health and of the health 
system itself, within which actions need to be taken, moni- 
tored, and managed. 

Systems thinking provides a means to understand and 
work with this complexity. It aims to gain insights into 
the whole by understanding the linkages, interactions, 
feedbacks, and processes between the elements that com- 
prise the whole system. In many cases, complexity stems 
from a combination of the complexity of the disease or 
condition itself (such as neonatal mortality) and the 
systems in which they are interacting and evolving, in this 
case the health system [8]. Health systems share the char- 
acteristics of complex adaptive systems [9]. They are 
constantly changing, tightly linked and governed by feed- 
back, hence constantly coping and adapting to actions or 
changes in other parts of the system. They are history 
dependent and therefore often resistant to change and 
new directions, especially those initiated by the stewards 
of the system [10,11]. Therefore, intervening in the system 
almost always has ripple effects that affect other parts of 
the system and introducing change is often not as straight- 
forward as the policy plans and design imply [11,12]. 
These are just a few of the reasons that argue for using a 
holistic systems thinking approach that takes into account 
this intricate complexity. 

This study aimed to contribute to this timely debate 
by exploring: i) how systems thinking tools, more specif- 
ically causal loop diagrams (CLDs) [13,14] and system 
dynamics modelling [15-17], can help better understand 
the complexity underlying the factors influencing neo- 
natal mortality, particularly in LMICs; and ii) what strat- 
egies and leverages may be successful in accelerating 
progress, using Uganda as a case study. The overall goal 
was to offer a comprehensive approach to examine the 
questions that can be applied in Uganda and can be 
adapted to other conditions, countries, and contexts. 


Methods 

This study employed the dynamic synthesis method- 
ology (DSM). DSM combines two powerful research 
strategies, namely the qualitative (case study research 
method) [18-20] and the quantitative techniques (simula- 
tion models) [15-17], to provide solutions to problems. 
Figure 2 presents the DSM by Williams (2000) [21], later 
revised by Rwashana and Williams (2009) [13]. DSM has 
six stages, namely i) problem statement and preliminary 
data collection; ii) field studies; iii) model building; iv) case 
study and empirical exploration; v) simulation; and vi) pol- 
icy analysis. This study applied the first three stages con- 
cluding with the development of a refined and validated 
CLD. The remaining three phases are underway and will 
be published subsequently. 
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Stage 1: problem statement 

Preliminary information related to neonatal mortality 
and the associated problems was collected from peer- 
reviewed literature as well as global and local reports 
and policy documents in order to understand and better 
characterize the current problems influencing neonatal 
health. The collected data included historical trends of 
mortality rates and coverage of key related interventions, 
as well as information describing quality of care, percep- 
tions, views, and attitudes of households and health 
workers in Uganda. 


Stage 2: field studies 

Semi-structured interviews were conducted with various 
stakeholders including mothers, village health workers, 
community leaders, front-line health workers at the 
first-level primary facilities and hospitals, and district 
and national policy and decision makers (see Additional 
file 1 for the data collection instruments). The purpose 
of the interviews was to explore the experiences, views, 
sources of information, and attitudes with respect to what 
happens during pre-conception, antenatal care (ANC), de- 
livery and postnatal care (PNC). It also elicited insights on 
quality and management of health services for women 
seeking to be future mothers, pregnant women, and neo- 
nates. Interview guides were tailored to each type of re- 
spondent to cover the relevant range of questions. They 
were informed by the information gathered during the 
first stage and brainstorming among the study team of 
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{21} revised by Rwashana and Williams (2009) [13]. 


other factors or problems that should be explored. They 
also allowed for exploring additional aspects or problems 
raised by the respondents through open ended questions. 


Study population and sampling criteria 

The interviews were conducted in Rubaga and Kawempe 
divisions of the district of Kampala, Uganda, where 47% 
of the population of Kampala reside. The Rubaga division 
was purposely chosen because it has two major mission 
hospitals that provide care for the low- and middle- 
income population from Kampala and nearby surrounding 
rural areas. Furthermore, Rubaga has the highest neonatal 
mortality rate of 54 per 1,000 live births in Uganda, com- 
pared to the national average of 27 per 1,000 [22]. The 
Kawempe division was chosen because it has the only 
national referral Hospital (Mulago) receiving referrals 
from all over the country. The sampling frame and se- 
lection criteria for the different interviews are presented 
in Table 1. 


Mothers 

A random sampling approach was used to identify mothers 
in antenatal clinics and from homes in the villages of 
Rubaga and Kawempe. At 95% confidence interval, a 
neonatal death rate (p) of 5.4%, and a level of permissible 
error (e) of <10%, the sample size was determined (see 
Additional file 2 for sample size calculation). The sample 
size was estimated as 282 mothers, divided equally between 
Rubaga and Kawempe (i.e., 141 each). In each division, 85 
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Table 1 Sampling frame and selection 
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Criteria — _ Comment 


Category Sample size Sa __ Sampling approach —_ 
Mothers a7 | 282 Random sampling 
Village health 16 VHWs and 10 Convenience sampling 


workers (VHWs) and community leaders approach 


community leaders 


Purposive sampling inclusive 
of both government and 
private health units in the 
two divisions. 


20 FHWs (13 nurse/ 
midwife/clinical 
officers and 7 doctors) 


Frontline health 
workers (FHWs) 


A purposive sampling 
approach. 


7 leaders/policy 
makers 


Healthcare decision 
and policy makers 


Overall, 274 interviews were 
considered; 8 had significant 
missing information 


Mothers who delivered in the last 
12 months residing in Rubaga and 
Kawempe divisions 

We stopped identifying new 
interviewees when no new 
issues were raised in the last 
interviews 


VHWs and community leaders 
residing in Rubaga and Kawempe 
divisions 


One staff was interviewed from 


Officers-in-charge of the facility and/or 
each available cadre/level 


health workers providing voluntary 
counselling and testing, or prevention 
of mother to child transmission services 


All the leaders/policy makers 
were interviewed 


Selected on the basis of their role in 
formulation and implementation of 
neonatal health policies 


of the 141 mothers were sampled from health facilities and 
56 from home. Mothers who delivered in the last 
12 months and had consented to the study were in- 
cluded in the study, regardless of whether their babies 
were alive or dead. Mothers whose last child was more 
than 12 months old or was mentally ill were not in- 
cluded. Although the planned interviews were 282, only 
274 interviews (Kawempe: 51 home, 88 health facilities; 
Rubaga: 55 home, 80 health facilities) were considered 
in the analysis. The remaining 8 interviews had signifi- 
cant missing information. 


Village health workers and community leaders 

A convenience sampling approach was used to select the 
respondents by selecting those residing in the villages 
where interviews with mothers and health workers oc- 
curred. A total of 16 village health workers and 10 commu- 
nity leaders from the villages of Rubaga and Kawempe 
were interviewed. We stopped identifying new interviewees 
when no new issues were raised in the last interviews. 


Front-line health workers 

Twenty front-line health workers were selected from 
Kawempe and Rubaga health facilities. The health facil- 
ities were purposively selected as follows: 


— Kawempe: Mulago National Referral Hospital, one 
private hospital, Kawempe government health centre 
IV and three private health centres. 

— Rubaga: Mengo hospital (mission hospital private 
not-for-profit), one private hospital, two private health 
Centres, and two government health centre III. 


During the interviews with the in-charge of the facilities, 
health workers providing voluntary counselling and test- 
ing or prevention of mother to child transmission services 
were identified. Among the health workers who are per- 
forming these services, one staff was interviewed from 
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each available cadre/level, purposively selected in discus- 
sion with the in-charge. The total sample included 13 
nurse/midwife/clinical officers and 7 doctors from public 
and private health centres and hospitals in both divisions. 


Healthcare decision and policy makers 

A purposive sampling approach was used to select 7 
leaders/policy makers, on the basis of their role in formu- 
lation and implementation of neonatal health policies. 
These include two heads of the obstetrics and gynaecology 
departments of hospitals in each of the selected divisions; 
two neonatologists, one in a private and one in a public 
hospital; one district health officer, the person in charge of 
women and children issues at the district headquarters; 
one division health officer; and one ministry official in- 
volved in neonatal health. 


Ethical consideration 

Ethical approval was obtained from Mengo Hospital Re- 
search Review Committee and the National Council of 
Science and Technology of Uganda. Consent forms were 
prepared to protect and ensure the dignity and welfare 
of all participants, as well as those who may be affected 
by the results of the research project. All participants 
were asked to sign a consent form and were informed 
that participation was voluntary and that they could opt 
out at any moment. Anonymity was ensured by using 
study identification numbers and initials rather than 
names of individuals. 

Mothers who were able to read and write filled in the 
questionnaire, while those who could not were inter- 
viewed. The socio-demographic characteristics of the 
mothers who interviewed for those are presented in 
Table 2. The age distribution, household income, and 
number of mothers who lost at least one neonate was 
similar to national average, while mothers’ education, 
percentage that were housewives, and the number of 
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Table 2 Socio-demographic characteristics of mothers 
interviewed during the field studies in two divisions of 
Kampala District, Uganda, n= 274 


Variable — - Category — n (Percentage) 
Age 15-20 49 (17.9) 
21-30 162 (59.1) 
31-40 59 (21.5) 
40+ 4 (1.5) 
Total 274 (100) 
Marital status Married 209 (76.4) 
Not married 64 (23.2) 
Widowed/divorced 1 (0.4) 
Total 274 (100) 
Highest level of education None 9 (3.3) 
P1—P7 72 (26.2) 
Secondary education 140 (51.7) 
Post-secondary education 51 (18.8) 
Total 274 (100) 
Occupation Farmer 5 (1.8) 
Housewife/does not work 122 (44.7) 
Health worker 9 (3.3) 
Teacher 13 (4.8) 
Business woman 94 (34.1) 
Other professions 31 (114) 
Total 274 (100) 
Household income Below 50,000 34 (12.4) 
ap leny eal 50,000-100,000 66 (24.1) 
Above 100,000 144 (52.6) 
Not indicated 30 (10.9) 
Total 274 (100) 
Number of pregnancies 1-3 204 (74.4) 
4-6 63 (23.1) 
7+ 7 (2.6) 
Total 274 (100) 
Had lost neonate Yes 18 (6.6) 
No 256 (93.43) 
Total 274 (100) 


pregnancies was closer to the urban rather than national 
rates [22,23]. 


Stage 3: model building and validation of the causal loop 
diagrams 

First, we pooled the different sources of data from stages 
one and two and categorised these according to the fol- 
lowing themes: factors associated with the mothers’ at- 
tendance to health services; social/personal characteristics 
associated with mothers’ attendance to health services; 
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factors that contribute as well destroy the health of neo- 
nates; factors associated with the health service delivery 
in health facilities; and factors in the community and 
family that are associated with mothers’ attendance to 
health services. 

We then used thematic analysis to compile and ana- 
lyse the qualitative data. Descriptive statistics and cross 
tabulations were used to explore the quantitative data. 
SPSS 10.0 was used for these analyses [24]. Using the 
findings, we brainstormed and generated a list of poten- 
tially important variables that are associated with neonatal 
mortality, which was used to develop descriptive CLDs 
using Vensim Software [25]. The full list of variables con- 
sidered for this analysis are presented in Additional file 3, 
which includes variables that were not considered in this 
analysis as well as evidence of association existing in the 
literature; this was not supported by our empirical find- 
ings for this case study. 


Development of casual loop diagrams 

CLDs help us to understand and depict the feedback 
mechanisms that are generated within complex systems 
which include the relationships, dynamics, and delays as- 
sociated with the variables that generate them. They offer 
a practical way to understand and express the systems’ in- 
terrelated parts and the cause-effect linkages for the prob- 
lem in question. CLDs are composed of two components; 
variables and influences (links). An influence has direction 
shown by an arrow and an indicator as to whether the in- 
fluenced element is changed in the same (+) or opposite 
(—) direction as the influencing element. That is, a link 
from element A to element B (4~~~~~~*8) may be posi- 
tive if a change in A produces a change in the same direc- 
tion, or negative (48) if a change in A produces 
a change in B in the opposite direction. A change in elem- 
ent A which produces a change in element B only after a 


delay is denoted byA cB. 


Feedback loops occur when arrows connect a variable 
to itself through a series of other variables. There are 
two main types of feedback loops that can be expressed 
using CLDs: balancing and reinforcing loops, as illus- 
trated in Figure 3. Balancing loops apply where there is 
an attempt to solve a problem or achieve a goal. They 
are also called neutralizing loops, where cause and effect 
cycles seek to counter a change with a push in the op- 
posite direction. Figure 3 shows a balancing loop whose 
goal is to increase the mothers’ participation in health 
services. As more mothers participate in health services, 
the workload increases, thus increasing the waiting 
times resulting in frustration, which in turn lowers the 
participation. Reinforcing loops represent a growing ac- 
tion where each action adds to another and may be re- 
ferred to as virtuous cycles when they produce desirable 
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Figure 3 Example of balancing and reinforcing loops. 


effects or vicious cycles when they produce negative ef- 
fects. Figure 3 also shows a reinforcing loop where grow- 
ing participation in health service arising from safe 
deliveries results in increased trust, which further in- 
creases participation. 


Validation of the causal loop diagrams 

Validation of the CLDs and the underlying conceptual 
thinking was carried out by nine local and international 
neonatal and maternal health stakeholders, including 
both researchers and implementers, as illustrated in 
Table 3. 

Respondents were asked to state whether all the vari- 
ables and relationships in the CLDs existed and whether 
there were any significant causal factors missing. In case 
there were some factors missing, they were asked to list 
them. In addition, respondents tested whether the direc- 
tions of each of the links were right or needed to be re- 
versed (implying that the effect is the cause) and were 
asked to state whether there were other effects that 
could be observed as a result of the causes in the CLDs. 
The validation instrument also included explanations of 
the objectives of the validation exercise, the meaning of 
the elements used in the CLDs, and how to respond to 
the questions; see Additional file 4 for the validation 


instrument. Suggested modifications from the validation 
exercise were discussed by the researchers and used to 
further improve the CLDs together with further qualita- 
tive analysis of the collected data in response to ques- 
tions raised by the validators, leading to the CLDs 
presented below. 


Findings 

Responses from all the interviews were categorized into 
demand and supply issues. The demand side captures 
the issues associated with the uptake of health services 
while the supply side shows the issues associated with 
the supply of health services. The key findings are pre- 
sented in turn below. 


Demand side issues 

Pre-conception 

Overall, 44% (8/18) of the mothers who lost their neo- 
nates suffered from diseases before pregnancy, including 
HIV, high blood pressure, malaria, sickle cell, and dia- 
betes, among others. 


Antenatal care 
While 97.8% of the mothers in our sample attended 
ANC at least once, 25.9% of them attended fewer times 


Table 3 Experts involved in the causal loop diagram validation (n = 9) 


Head of Paediatrics Department Mengo Hospital, Uganda 1 
Obstetrician and Gynaecologist Mulago Referral Hospital, Uganda 2 
Lecturer, Department of Obstetrics and Gynaecology College of Health Sciences, Makerere University, Uganda 1 
Nursing Officer in-charge of Maternity Department Mengo Hospital, Uganda 1 
Head, Obstetrician and Gynaecologist Mengo Hospital, Uganda 1 
Paediatrician and researcher in maternal, neonatal, and child health issues Universidad Peruana Cayetano Heredia and Universidad 1 
Nacional Mayor de San Marcos, Lima, Pert 
Paediatrician/Professor Department of Global Public Health and Primary Care, 1 
University of Bergen, Norway 
Paediatrician/Neonatologist WHO, Coordinator, of the maternal, neonatal and child 1 
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than recommended, providing the following reasons for 
failure to attend: lack of money for transport, busy work 
schedules, attending school, and delayed start of ANC 
clinic. 


Delivery 

The majority of the mothers (97.4%) were provided 
transport to the health centre for deliveries by the com- 
munity (56.9% spouse; 17.2% relatives; 9.0% friends, and 
3.0% neighbours). Only 13.1% used their money for 
transport. 


Postnatal care 

Some of the respondents lacked knowledge on how to 
care for the babies. For example respondents stated that 
they used vaseline, herbs, and powder for cord care. 
Some used breast milk, herbs, urine, water, and saline to 
care for swollen eyes. 


Attitudes and beliefs 

While almost all mothers (97.4%) in our sample gave 
birth at a health facility and believed in the importance 
of doing so, 40.9% stated that they knew of mothers in 
their community who did not go to health facilities for 
their deliveries. Cited reasons for why some of these 
mothers may have chosen not to go to health facility for 
their deliveries included traditional beliefs (14.1%), reli- 
gious beliefs (3.2%), lack of permission from the spouse 
(3.8%), and lack of trust in the health system (14.7%). 
On the latter, more elaborations from the mothers in- 
cluded that they knew of mothers who feared that health 
facilities were poorly equipped, had insufficient health 
workers, were overcrowded with long waiting lines, 
lacked 24-hour care, had careless, rude, and abusive 
health workers who carried out excessive episiotomy, 
and that they found seeking care at health facilities to be 
very costly, possibly due to unavailability of drugs and 
lack of free supplies at health centres such as the Free 
Mama Kits for those who do not have their own. 


Sources of information 

Mothers stated that they generally obtain health infor- 
mation from various sources including radio, friends, 
brochures, films, health workers, family, and newspapers. 
With respect to information provided to the mothers 
during ANC, 92.8% of the mothers received information 
on HIV and the value of HIV testing, 84.6% on family 
planning, 84.3% on breast feeding, and 83.2% on nutri- 
tion (83.2%). When asked about what they thought were 
the best methods to encourage mothers to attend ANCs 
and give birth at health facilities, they listed the follow- 
ing: home visits by health workers, village meetings, so- 
cial meetings, community notices, and health education 
during ANC sessions. 
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Supply side issues 

Quality of health service and hygiene 

Although several mothers perceived hygiene at the 
health facilities to be generally good, 71% of the mothers 
reported that hygiene was still in need of further im- 
provement. The surveyors also observed that some of 
the facilities were not well maintained and infection pre- 
vention was not well observed. Community leaders also 
noted that there were cases of neglect by health workers; 
one leader said “... babies born with ... no one to wrap 
them”. 


Health workers motivation 

All of the interviewed health workers mentioned that they 
were poorly paid. It was also observed by the mothers and 
the surveyors that staff experienced burnout due to work- 
load arising from few skilled workers at the units resulting 
in a high provider to number of deliveries ratio. Health 
workers stated that both remuneration and safety measures 
at the health facility would increase their motivation. They 
also stated other demotivating problems such as electric 
power supply breakdowns during delivery, mothers losing 
a lot of blood before or after delivery especially where no 
blood bags were available, and difficulty in getting mothers 
to a referral hospital in case of emergency due to lack of 
ambulances. 


Availability of supplies and equipment 

Staff voiced frustration due to unavailability of the neces- 
sary equipment and supplies. According to our survey, 
34% of health workers had deficiencies in resuscitation 
equipment and 67% lacked ultrasound facilities. 


Record keeping 

Only 16.8% of the health workers in our sample pre- 
pared births and death certificates and 83% kept Health 
Management Information System child health records 
and submitted them to higher levels; 50% of the village 
health workers reported that they lacked birth registers. 


Policy enforcement 

Community leaders and policy makers noted that some 
policies and guidelines were not readily available to the 
public and were not always adhered to. Only 33% of the 
health facilities had clinical guidelines available to them. 
For example, guidelines on cord care were not readily 
available. It was also noted by the policy maker at the 
district level that there was no clear policy and enforce- 
ment on recruitment of appropriate numbers of front- 
line health workers for the population. 


Supervision 


Overall, 18.8% of the village health workers reported that 
they were not supervised, which is consistent with 
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community leaders perceptions of gaps in supervision at 
the community level. Similarly, supervision of health 
workers at health units was also considered poorly 
enforced. In addition, community leaders voiced con- 
cerns about the number of unqualified people treating 
the population in their communities and that efforts to 
control this health risk should be strengthened. 


Causal loop diagrams 

Two CLDs depicting the factors associated with the de- 
mand for (Figure 4), and supply of (Figure 5) health services 
for neonates and mothers were created from the interview 
and the data collection in stage one, together with brain- 
storming among the study authors. Several reinforcing and 
balancing feedback loops can be observed in these CLDs. A 
detailed analysis of the CLDs is provided below: 


Dynamics of the demand for neonatal and maternal 
healthcare service 

The dynamics involved in the demand for neonatal and 
maternal healthcare are presented in Figure 4. We 
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identified one balancing loop (B1, frustration loop) 
where there is an attempt to achieve the goal of increas- 
ing uptake of maternal health service and four reinfor- 
cing loops (R1, awareness loop; R2, trust loop; R3, myths 
loop; and R4, health loop) that represent growing actions 
as illustrated below. 

The awareness loop (R1) is a virtuous cycle that en- 
hances the growth of awareness. The level of awareness 
of neonatal and maternal health issues results in im- 
proved health of mothers and increased attendance to 
neonatal and maternal healthcare services, thereby low- 
ering the death risk of neonates. Awareness is enhanced 
through health education programmes provided during 
ANC visits, media (TV, radio, newspapers), sensitization 
by community leaders, and word of mouth through peer 
to peer interactions among mothers. As mothers attend 
ANC, PNC, and hospital deliveries, the level of aware- 
ness increases resulting in mothers’ preparedness for birth. 
Mothers’ birth preparedness, which is achieved with in- 
creased level of awareness, family and community sup- 
port, and socio-economic status increases the likelihood 
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of attending health services and therefore having safer de- 
liveries, and further enhances the attendance to ANC. The 
growth in the awareness loop eventually slows down due 
to the inadequacy of resources exhibited in the frustration 
loop resulting into the limits to growth archetype (loops 
Ri and Bl). In order to avoid the limits to growth, the 
quality of service in the health facilities must be sustained. 

The trust loop (R2) enhances the trust of women in 
health systems through provision of safe health care de- 
liveries and PNC. As more mothers attend ANC, hos- 
pital deliveries, and PNC, the level of safe deliveries and 
PNC increases, which in turn increases their trust in the 
healthcare service. The growth in the trust loop eventu- 
ally encounters limiting action thereby exhibiting the 
limits to growth archetype (loops R2 and B1). The limits 
to growth of this cycle arises from inadequate resources 
that are needed to sustain the quality of maternal and 
neonatal healthcare service deliveries as exhibited in the 
frustration loop (B1) explained below. In order to main- 
tain the trust, the quality of the maternal and neonatal 
service must be observed. 


The frustration loop (B1) shows that the desired state 
is to have as many mothers attending ANC, healthcare 
deliveries, and PNC. Attendance to ANC and delivering 
at health facilities plays a big role in promoting safe de- 
liveries and obtaining PNC, which will also contribute to 
increased trust in the health system and improving the 
general awareness about the benefits of these health ser- 
vices in the community. When the number of women 
participating in maternal and neonatal health services 
increases, the resources (staff, drugs, logistics, and sup- 
plies) in the health facilities are depleted, leading to frus- 
tration resulting from effects of poor service delivery 
such as long waiting times and drug stock outs, which 
results in a decrease in attendance, thus demonstrating a 
balancing loop. Efforts should be made to ensure that 
the resources in the health facilities match the demand, 
thereby minimising frustration of mothers. 

The myths loop (R3) produces a desirable effect 
whereby beliefs in myths are decreasing. As the level of 
awareness on maternal! health care (MHC) and neonatal 
health care (NHC) increases, belief in perception and 
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myths decreases. As the belief in perceptions and myths 
decreases, the level of awareness increases. Belief in 
myths and perceptions that are enhanced as a result of 
low maternal literacy levels are a hindrance to the level 
of awareness. Efforts to keep the growth of awareness 
through community and peer to peer sensitization, health 
education, and media should be made so that eventually 
the myths die off. 

The healthy mothers loop (R4) produces a virtuous 
cycle where mothers’ attendance to ANC and hospital 
deliveries results in improved mothers’ health, thereby 
producing safe deliveries, which builds trust resulting in 
a further increase in the mothers’ uptaking of health ser- 
vices. This loop interacts with the frustration loop creat- 
ing the limits to growth archetype. 

The CLD shows that neonatal health heavily depends 
on the health of the mothers. The health of the mothers 
can be increased by increased self and household hy- 
giene, increased level of awareness, attendance to ANC, 
PNC, and health facility deliveries, and adherence to the 
recommended nutrition. Factors that lower the mothers’ 
health include increased frequency of child delivery, 
diseases such as malaria, and teenage pregnancies, 
among others. The death risk of a neonate increases 
with hypothermia, poor breast feeding practices, poor 
socioeconomic status, and poor care of the neonate 
resulting from lack of awareness. 


Dynamics of the supply for neonatal and maternal 
healthcare service 

The dynamics involved in the supply of neonatal and 
maternal health services are presented in Figure 5. In 
this CLD, one reinforcing loop (R5, motivation loop) 
that represents a growing action in the motivation of the 
workforce and four balancing loops (B2, transport loop; 
B3, workforce loop; B4, logistics loop; and B5, workload 
loop) all representing desired goals towards improve- 
ment in the supply of maternal health service are identi- 
fied and explained below. 

The transport loop (B2) emphasizes the importance of 
having timely and adequate referrals between hospitals 
towards improvement of the maternal and neonatal 
healthcare services, which can be achieved through the 
provision of transport/ambulances. As the funding for 
maternal and neonatal healthcare increases, provision of 
ambulances and transport increases, resulting in timely 
and adequate referrals between health units. This im- 
proves maternal and neonatal healthcare service delivery 
which lowers the death risk of neonates and in turn 
lowers neonatal mortality rates. A rise in neonatal mor- 
tality rates attracts an increase in advocacy initiatives 
resulting in increased funding. Funding for purchase and 
maintenance of vehicles as well as policies for use of 
these vehicles should be done. 


100 — HEALTH RESEARCH POLICY AND SYSTEMS 


Page 10 of 14 


The workforce loop (B3) shows that the desired state 
is to have a motivated workforce in terms of having the 
right skills, remuneration, and attitude. An increase in 
the funding for maternal and neonatal healthcare results 
in an increase in the training, recruitment, and remuner- 
ation of health workers, resulting in a motivated work- 
force. This improves maternal and neonatal healthcare 
service delivery, which lowers the death risk of neonates 
and in turn lowers neonatal mortality rates. A rise in 
neonatal mortality rates attracts an increase in advocacy 
initiatives resulting in increased funding. For this to hap- 
pen, governments must be willing to fund and invest in 
the training, recruitment, and remuneration of health 
workers. Failure to do so eventually results in poor ma- 
ternal and neonatal healthcare, thereby increasing the 
death risks of neonates and resulting in increased neo- 
natal mortality. 

The logistics loop (B4) shows that the desired state is 
to have quality maternal and neonatal health services 
where the health facilities have logistics, drugs, and re- 
suscitation kits. As the funding for maternal and neo- 
natal healthcare increases, provision of infrastructure, 
diagnostic and resuscitation kits, drugs, and logistics and 
logistical support increase. This improves maternal and 
neonatal healthcare service delivery, which lowers the 
death risk of neonates and in turn lowers neonatal mor- 
tality rates. A rise in neonatal mortality rates attracts an 
increase in advocacy initiatives resulting in increased 
funding. Governments should provide adequate funding 
to ensure that the required resources are made available. 

The motivation loop (R5), together with the workload 
loop (B5), make up a limits to growth archetype. Limits 
to growth of this loop result from exceeding the capacity 
of mothers that can be handled by the workforce. The 
motivation loop shows that a motivated workforce that 
is well supervised, remunerated, and trained, and with 
adequate supplies will increase the maternal and neo- 
natal healthcare service delivery. When the service deliv- 
ery is good, this, in turn, further motivates the staff 
resulting in a virtuous cycle. The workload loop (B5), on 
the other hand, shows that an increase in a motivated 
workforce increases the number of mothers attending 
ANC, hospital deliveries, and PNC, which in turn in- 
creases the workload thus lowering the workforce that is 
motivated. It is therefore important that the health ser- 
vices and workforce are upgraded to meet the growing 
population which will even out the number of patients 
attending the few health facilities. 

With adequate financing of maternal and neonatal 
health services, the following can be made available: 
training, recruitment, and adequate remuneration of 
health workers, resulting in a motivated workforce pro- 
viding quality healthcare service and thus leading to safe 
deliveries; transport for timely and adequate referrals 
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between health units and obtaining blood bags if neces- 
sary; and equipment (e.g., for resuscitation, suction or 
oxygen), medical supplies, emergency drugs, and safe de- 
livery kits. The availability of funding coupled with good 
governance is necessary for the formulation and enforce- 
ment of healthcare guidelines, planning, supervision, 
and efficient and equitable resource allocation, as well as 
monitoring and evaluation and audits of health facilities. 

Although the graph in Figure 1 demonstrates that the 
skilled birth attendance has increased slightly there has 
not been significant decline in neonatal mortality rates 
over the past two decades. This clearly brings out the 
various limits to growth loops identified in the CLDs, 
which show that as the limits to growth are reached, the 
growth engine loses its effectiveness, and the growth 
curve begins to flatten. 


Validation of the causal loop diagrams 

Respondents used the validation instrument in Additional 
file 4 to test the CLDs. The validation process contributed 
to further modification and led to the final CLDs pre- 
sented in this paper. The respondents were also asked to 
generally assess their experience with viewing the issues 
related to neonatal health by means of the CLDs. All the 
respondents saw and understood the CLD for the first 
time. The objective was to determine whether they found 
the CLDs to be reasonable, representative of the health- 
care issues, and whether they were useful aid and commu- 
nication tools. See Table 4 for their perceptions. 


Table 4 Overall impressions of the experts regarding the 
CLDs 
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Discussion 

This study presents the first of its kind in-depth analysis 
of the possible causes of neonatal mortality in a given 
context with an explicit focus on complexity. We expli- 
citly examined the feedback loops that were generated 
due to the complexities surrounding neonatal mortality as 
a first step towards considering and testing alternative 
short- and long-term strategies that may be used to effi- 
ciently address the root causes of some of these problems. 

One of the main points of strength in this study is the 
inclusion of a wide range of perspectives of the different 
key stakeholders, including the mothers, front-line health 
workers, and village health workers. This undoubtedly 
enriched the analysis and provided a deeper understand- 
ing of the real causes of neonatal mortality and its inter- 
play with the complexities of the health system it interacts 
with [9]. The second strength of this paper is using a val- 
idation instrument to validate the CLDs, which strength- 
ened our model and its global relevance, given that we 
also approached international stakeholders from various 
regions of the world. 

A limitation of this study is that it was undertaken in 
Kampala district. It is possible that there are other fac- 
tors that are peculiar to other parts of Uganda such as 
the geographical environment (terrain), extreme poverty, 
and rural levels that have not been captured in this re- 
search. Our findings may, therefore, be more representa- 
tive of urban Uganda than the whole country. However, 
this is only relevant for quantitative studies. The main 
objective of our study was to exploit the strength of 
qualitative approaches to explore how this problem, and 
its intricate complexities, can be understood in-depth, 
using a systems thinking approach. The objective of our 
study, therefore, included a methodological component 
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ANC at least once and more than 80% of them had good 
knowledge about the main health problems that are rele- 
vant to pregnancy and labour. 

Our findings also highlighted the importance of sup- 
portive spouses and community involvement and _ its 
contribution to higher utilization of health services, es- 
pecially for delivery, where the majority of the women 
(97.4%) received support by the community and only 
13.1% used their money for transport. Our findings also 
highlighted health system problems that are standing 
against any possible improvement in neonatal mortality. 
These include the situation of health facilities in terms 
of hygiene and infection control, lack of cheap life- 
saving equipment and supplies, such as resuscitation 
kits, and suction machines. 


Leverages 

Leverages are influences within the system where small 
changes can effect a substantial change in the system. 
From the analysis of the CLDs, the following were per- 
ceived as high leverage points which can effect signifi- 
cant improvement in neonatal healthcare: 


e Increased awareness on maternal and neonatal 
healthcare can weaken the vicious cycle exhibited by 
the myths loop (R3) while strengthening the 
virtuous cycle of the awareness loop (R1). Mothers’ 
awareness on the recommended feeding, nutrition, 
hygiene, household environment, and mothers’ birth 
preparedness and efforts to avoid untreated diseases 
results in improved health of the mothers, which in 
turn lowers neonatal mortality rates. Some of the 
short-term interventions which may improve 
awareness include aggressive advertising, campaigns, 
sensitization, and education of the women and girl 
child as well as increasing the effectiveness of the 
health education sessions during ANC and PNC. 
Special gender considerations to ensure that girls 
receive essential education thereby increasing 
maternal mortality rates is a longer term strategy 
but would synergistically address many other health ’ 
and non-health issues. 

e The low socio-economic status is a key determinant 
in the health of the mothers and the neonates. With 
improved socio-economic status, mothers are able 
to obtain the recommended nutrition, healthcare, 
and the requirements for birth preparedness. While 
introducing incentives, such as transport vouchers 
and free birth kits for pregnant women, would 
motivate them to attend ANCs and enable them 
to be better prepared for health facility deliveries 
in the short term, the government should work 
towards improving the socioeconomic status of 
the nation. 
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e Funding for maternal and neonatal health care 
should be prioritized at the national level. Efforts by 
the government and policy makers to upgrade the 
health service infrastructure as well as build systems 
for monitoring the resources (staffing, drugs, and 
stocks) would go a long way in minimizing the 
effects arising out of the frustration loop B1. 
Improved maternal and neonatal health service 
delivery will strengthen the virtuous cycle created by 
the motivation loop R5. In addition, without a 
motivated health work force that is well trained, 
adequately remunerated, and with an acceptable 
workload there is not much to be expected in terms 
of the quality of the care provided nor the likelihood 
that mothers will come to seek care at health 
facilities. Other short- and long-term strategies may 
include improved supervision and internal audits at 
health facilities to ensure that maternal and neonatal 
guidelines are adhered to as well as establish the 
current conditions and gaps in resources (human, 
logistics, and drugs) to guide the funding for 
national health care. 


This study reports on stages 1~3 of our research de- 
sign. Future work involves the completion of stages 4—6, 
where we will use empirical data to develop the quanti- 
tative (simulation) model including testing of different 
policy options. Iterations to test and validate the model 
will be conducted through brainstorming sessions with 
stakeholders. What-if analysis will be used to test differ- 
ent strategies that have been suggested by this research 
and by stakeholders, including policy makers, during 
brainstorming and validation workshops with the aim of 
improving policy analysis and design in neonatal health. 
The model will be used to determine the strategies that 
could have a great impact on neonatal mortality using 
sensitivity analysis. 


Conclusions 

This study adopted a systems thinking approach to cap- 
ture and analyse the interactions between technical, pol- 
icy, behavioural, and cultural issues related to neonatal 
mortality. It provides a broad integrated view of the dy- 
namics associated with neonatal health, thus accommo- 
dating the different stakeholder viewpoints. The synthesis 
of the various theoretical concepts through the use of 
CLDs facilitated the understanding and interpretation of 
the different interacting elements and feedback loops that 
contributed to the stagnant neonatal mortality rates in 
Uganda, which is the first step towards discussing and ex- 
ploring the pros and cons of the different strategies and 
the priorities that should be addressed based on their 
likely impact and cost-effectiveness. 
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This paper also illustrated the importance of validation 
of the structure and relationships of the CLD with key 
stakeholders, including decision makers, which was 
beneficial, enriching, and ensured that the variables of 
the CLDs represent that of the real system. The valid- 
ation exercise demonstrated that CLDs can help the dif- 
ferent stakeholders view complex health problems from 
different perspectives and facilitate shared understanding 
and common ownership of the interpretations of health 
problems. They also provide a broad integrated view of 
the problems which can be used for learning and process 
improvement, as well as operational management. The 
methods, approaches, and findings from this study are 
not only applicable to neonatal health and Uganda, but 
also to other settings and questions of a similar nature. 
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Abstract 


Background: Governing immunization services in a way that achieves and maintains desired population coverage 
levels is complex as it involves interactions of multiple actors and contexts. In one of the Indian states, Kerala, after 
routine immunization had reached high coverage in the late 1990s, it started to decline in some of the districts. 
This paper describes an application of complex adaptive systems theory and methods to understand and explain 
the phenomena underlying unexpected changes in vaccination coverage. 


Methods: We used qualitative methods to explore the factors underlying changes in vaccination coverage in two 
districts in Kerala, one with high and one with low coverage. Content analysis was guided by features inherent to 
complex adaptive systems such as phase transitions, feedback, path dependence, and self-organization. Causal loop 
diagrams were developed to depict the interactions among actors and critical events that influenced the changes 
in vaccination coverage. 


Results: We identified various complex adaptive system phenomena that influenced the change in vaccination 
coverage levels in the two districts. Phase transition describes how initial acceptability to vaccination is replaced by 
a resistance in northern Kerala, which involved new actors; actors attempting to regain acceptability and others 
who countered it created several feedback loops. We also describe how the authorities have responded to 
declining immunization coverage and its impact on vaccine acceptability in the context of certain highly connected 
actors playing disproportionate influence over household vaccination decisions. 

Theoretical exposition of our findings reveals the important role of trust in health workers and institutions that 
shape the interactions of actors leading to complex adaptive system phenomena. 


Conclusions: As illustrated in this study, a complex adaptive system lens helps to uncover the ‘real’ drivers for 
change. This approach assists researchers and decision makers to systematically explore the driving forces and 
factors in each setting and develop appropriate and timely strategies to address them. The study calls for greater 
consideration of dynamics of vaccine acceptability while formulating immunization policies and program strategies. 
The analytical approaches adopted in this study are not only applicable to immunization or Kerala but to all 
complex interventions, health systems problems, and contexts. 


Keywords: Causal loop diagram, Complex adaptive systems, Governance, Immunization, India, Kerala, Systems 
thinking, Trust, Vensim 
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Background 
Organizing immunization services to protect the society 
against preventable diseases is a core function of public 
health. In India, the Universal Immunization Programme 
(UIP) introduced in 1985, targets 27 million infants and 30 
million pregnant women every year and is one of the largest 
in the world [1]. Though UIP has improved the availability 
of vaccines and cold chain management compared to earlier 
immunization programs, the system has not yet achieved 
sustained improvement in vaccination coverage in many 
Indian states [2,3]. It has been slated as a mechanistic 
approach, which was simplistically expected to improve 
immunization coverage through the improvements in health 
infrastructure, financing, supplies, and better management 
practices [4,5]. This approach has typically failed to account 
for the unique characteristics, interactions, and needs within 
local systems and the diversity of actors impacting a house- 
hold’s decision to vaccinate. Such an approach was often 
constrained by a lack of understanding of the complex 
behaviour of local health systems, which often do not 
respond as expected to external interventions and policies. 
Furthermore, such an approach can only provide a limited 
explanation for fluctuations in immunization coverage 
rates, over time. 

This paper describes an application of systems think- 
ing to understand the complex phenomena underlying 
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changes in vaccination coverage in India. Specifically, 
this study seeks to use a complex adaptive system (CAS) 
lens to understand the features of a complex system that 
governs childhood immunization in parts of the Indian 
state of Kerala, where immunization coverage drastically 
declined after a period of high coverage. 

Kerala holds a special place in the global public health 
discourse for its remarkable health achievements despite 
low economic status [6]. Unlike the national average of 
full immunization coverage (of BCG, Polio, DPT and 
measles) of 54.2%, Kerala had achieved over 84% in the 
late 1990s before starting to decline in subsequent years 
(Figure 1) [7]. This decline mostly involved the northern 
districts of Kerala. The reduction of coverage in north- 
ern districts in Kerala is a concern for public health au- 
thorities as it negates herd immunity that protected 
communities against the potential spread of vaccine- 
preventable diseases [8,9]. The sudden decline in 
immunization coverage, in a state where vaccines were 
uncritically accepted as a social good in the past, has 
puzzled public health officials and experts [10]. 


Theoretical underpinnings 

Our study adopted a CAS-lens for exploring the factors 
that influenced immunisation services in Kerala and 
identified many characteristics of the CAS phenomena 
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in the discussion. CAS exhibit the capacity to self-organize, 
adapt, and learn from past experiences, which often re- 
sults in counterintuitive and unintended effects or para- 
doxical behaviour [11,12]. CAS may exhibit various 
unique features, such as path dependence, feedback 
loops, scale-free networks, emergent behaviour, and phase 
transition. 

Path dependence refers to non-reversible processes that 
have similar starting points yet lead to different outcomes 
due to different choices made along the way. Feedback 
loops that feed into the system explain how small changes 
can grow into large consequences that can be ‘reinforcing’ 
a particular outcome or may lead the system to go back to 
an original state, what is called ‘balancing’ or goal-seeking 
loops. Phase transitions occur at tipping points when sys- 
tems show sudden changes even without any additional 
inputs. Scale-free networks refer to the formation of influ- 
ential hubs that can shift the focus and power of networks 
by exerting higher influence on other actors in the net- 
work through their multiple interconnectedness, hence 
activating a change in behaviour. CAS can also exhibit 
emergent behaviour when smaller entities jointly create a 
spontaneous order and show organised behaviour. 

In the context of the governance of public health inter- 
ventions, such as immunizations, complexity is generated 
from the diverse and dynamic nature of interactions be- 
tween the system’s actors and the multiple interacting fac- 
tors such as values, culture, history, norms, and distribution 
of power and information in societies. Furthermore, the 
interconnectedness between the various actors of the sys- 
tem means that their actions inevitably create ripple effects 
that cause continuous change and adaptation of the inter- 
vention in the entire system [13]. 

Exploring these complex system features in Kerala will 
shed light on the factors that drove the unexpected 
changes in immunization coverage and will provide in- 
sights into the types of system adaptations that must be 
considered by national immunization programs. 


Methods 

A qualitative case study design was used to obtain an un- 
derstanding of immunization coverage in Kerala. The data 
used in this paper originated from a larger study seeking 
to understand the governance of immunization in two 
states in India [14]. Initial findings from this earlier study 
highlighted the need to further explore the complexity 
of immunization services in one of the states — Kerala — 
where we observed surprising trends in immunization 
coverage (Figure 1). 

Two districts in Kerala were randomly identified from 
high and low coverage districts in the state [7]. The dis- 
tricts were Alappuzha, a better performing (90.2%) dis- 
trict in terms of immunisation coverage as per the third 
District Level Health Service survey, and Kozhikode, a 
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poor performing (65%) district. In each of these districts, 
better performing and poor performing areas in terms of 
immunisation service were identified with the help of dis- 
trict level managers. Though the difference in immunization 
coverage between two areas within the districts was mar- 
ginal, the identification of different locations helped in col- 
lecting information from diverse contexts. From each 
area, two primary health centres and a private health facil- 
ity were selected for observation of the immunisation ser- 
vices and interviewing the practitioners. 

The main data sources included a literature and docu- 
ment review (including news reports), in-depth interviews, 
focus group discussions, and observations of immunization 
services. All data collection was undertaken by the first 
author in Malayalam (local language), over a period of six 
months during late 2009 and early 2010. 

In each district, in-depth interviews were carried out 
with immunization service providers from public and pri- 
vate sector, those who facilitate vaccination, such as com- 
munity health workers, and those who opposed it. We 
used a snowball sampling method, whereby, at the end of 
the interview, the respondent’s suggestion was asked about 
other important stakeholders in order to identify the next 
respondent. The experts interviewed were also identified 
using a snowball method based on their research experi- 
ence on immunization or their expertise of immunization 
service, either as a past or present state or district level 
immunization program implementer. 

Focus group discussions were undertaken with mothers 
of children below five years of age and one with the health 
workers of one of the primary health centres. The mothers 
were identified and invited with the help of community 
workers and the discussions were arranged in one of the 
local houses or local Anganwadies (pre-school and nutri- 
tion centres for women and children). The number of par- 
ticipants per focus group discussion varied from 7 to 10. 

Participant and non-participant observations were 
made with the help of an observation guide in order to 
gather insights into cultural meanings and interpreta- 
tions related to provider and beneficiary behaviours and 
context. All participant observations were made during 
the house visits that the first author made along with 
community health workers, aiming to mobilizing benefi- 
ciaries for vaccination. During each of the visits the re- 
searcher was introduced to households as a public health 
researcher and was involved in motivating and educating 
the families on childhood vaccinations. In most of the 
households visited, the initial communication related to 
vaccination was provided by the community health worker 
and the researcher was asked to clarify when further ex- 
planation was required. For participant observation, the re- 
searcher had to play the role of a public health expert and 
researcher simultaneously. This involved active engage- 
ment in mobilizing the parents for immunization of their 
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children along with making qualitative observations from 
this engagement for the research. Non-participant obser- 
vations were made during immunisation sessions at health 
facilities, outreach immunisation sessions, and review 
meetings of field staff in charge of the immunisation 
programme. Important observations were noted onsite 
and, at the end of the day, a full record of the field notes 
was prepared by appropriately commenting on each of 
these activities. 

All interviews and focus group discussions were digitally 
recorded, transcribed, and translated into English. Content 
analysis was applied to the transcripts of interviews and 
focus group discussions, as well as the field notes of obser- 
vations [15]. The various categories for content analysis, 
as informed by the application of a CAS lens, were identi- 
fied prior to the analysis. Using these categories, we used a 
deductive coding of the data. Atlas.ti ver.7 was used for ar- 
ranging the text according to codes and managing the 
codes in the interpretive phase. 

The three different methods were used for data collec- 
tion from various types of respondents; observation, inter- 
views, and focus groups involving various sources of 
information helped in triangulating the findings. In order 
to reduce the subjective bias of the first and second authors 
due to their prior information of the functioning of Kerala’s 
health system, a peer scrutiny of the analysis was done by 
the third author who assessed the assumptions made. 

Based on the qualitative data analysis, a causal loop dia- 
gram (CLD) was developed using Vensim PLE [13,16]. 
CLDs are qualitative representations of underlying mental 
models and are typically used to illustrate feedback and in- 
teractions among health system actors [17]. For this study, 
the purpose of the CLD was to assist in the identification 
and interpretation of the feedback loops that emerged in 
the context of immunization. The CLD was also used to 
guide the brainstorming discussion among authors about 
other complex phenomena that governed the analysis 
period. The variables used in the CLD were derived from 
the qualitative data, as well as from the literature on de- 
terminants of immunization coverage. The CLD uses 
standard notation, where positive arrows denote that 
two variables change in the same direction, and negative 
arrows denote that two variables change in opposite direc- 
tions. An arrow with a double hash mark on it (||) indi- 
cates that there is a time delay in the relationship denoted. 
Reinforcing loops, which indicate that variables have an 
overall amplifying effect, are labelled with an “R” and a 
loop symbol. Balancing loops, which indicate that vari- 
ables have an overall dampening effect, are labelled with a 
“B” and a loop symbol. The loop symbol is either clock- 
wise or counter-clockwise, depending on the direction in 
which the loop is read [17]. Where there were multiple 
loops, we numbered them in the order in which they ap- 
pear in the text. We used the CLDs not only as a summary 
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of the content analysis, but also to conceptualize and de- 
velop additional potential linkages between factors. Dotted 
arrows have been used to denote those potential add- 
itional relationships that were not empirically explored. 

The study protocol was reviewed for ethical and tech- 
nical clearance by the Institutional Review Board (Sree 
Chitra Tirunal Institute for Medical Science and Tech- 
nology, Thiruvananthapuram, India). Written permission 
for data collection was obtained from state level health 
officials as well as from district level officials and partici- 
pation in the study was made voluntary by ensuring in- 
formed consent from all participants and the possibility 
to withdraw at any time. All identifiers of the study par- 
ticipants from the transcripts of the data were removed 
by the first author to ensure anonymity of the study 
participants. 


Results 

The fieldwork included 7 participant and 7 non-participant 
multi-site observations, 5 focus group discussions, and 
17 interviews with beneficiaries, community intermediar- 
ies (community health workers, nutrition and pre-school 
teachers and community leaders), and providers from 
public and private sector. The study also involved key in- 
formant interviews with 6 experts. 

As described in the introductory section, our analysis of 
trends in immunization coverage in both districts showed 
a sudden decline in immunization coverage in Kozhikode; 
based on three rounds of the District Level Household 
and Facility Survey, Kozhikode showed a decline after the 
second round of the survey in the 2002-2004. period. The 
full immunization coverage in Kozhikode district in north- 
ern Kerala dropped from 94% (2002-2004) to 65% (2007— 
2008). During the same period, the coverage in a southern 
district, Alappuzha, had in fact gone up from about 84% 
to around 92%. The decline in immunization coverage in 
Kozhikode is, in fact, a reversal of the trend from the earl- 
ier period between the first (1998-1999) and the second 
survey (2002-2004), which showed improvement in vac- 
cination coverage. 

The qualitative data showed a widespread hesitancy 
against routine vaccinations in Kozhikode district, while 
routine vaccinations are widely accepted in Alappuzha 
district. It was also observed that the resistance against 
vaccination was often limited to geographical locations. It 
was observed during the house visits in vaccine-resistant 
areas of Kozhikode district that most of the unvaccinated 
children are found in households of close geographical 
vicinities. The differences in immunization coverage in 
different areas within Kozhikode district is explained by 
the spread and extent of vaccine-resistant geographical 
locations within the district. However, we could not 
elicit major differences in vaccine acceptability between 
high and low coverage areas of Alappuzha district, which 
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may be explained by other factors such as poor socio- 
economic status of the region, absence of public health 
human resources, or anomalies in the reporting of vaccin- 
ation coverage. 

In the following section, we first illustrate the feedback 
loops that emerged as a result of interactions among the 
key actors and contributed to phase transitions from vac- 
cination acceptance to resistance. We introduce two sep- 
arate CLDs that are relevant to the acceptability phase and 
vaccine-resistance phase to discuss the contrasting fea- 
tures of these two phases. We also describe the feedback 
that affected the districts differently after showing a high 
level of vaccine acceptability in the beginning. Next, we 
show how the authorities have responded to this problem 
of decline in immunization coverage and discuss the im- 
pact of their response in the presence of certain highly 
connected actors playing a disproportionate influence over 
a household's vaccination decision. 


Phase 1: Acceptability 

As mentioned in the introduction, the UIP heralded a shift 
compared to earlier programs in both the availability and 
acceptability of immunizations in Kerala. After a decade of 
implementation, society perceived vaccines to be effective 
in the prevention of certain diseases and coverage increased 
significantly. Figure 2 displays the CLD illustrating the fac- 
tors promoting the acceptability of immunization under 
UIP. There were several actors who contributed to this. 
Public allopathic doctors were important sources of health 
education and encouraged immunization. Private sector 
allopathic doctors also contributed to this effort either in 
collaboration with UIP or through their independent ef- 
forts. A large part of the success of the UIP program during 
this period was credited to the joint efforts of health field 
workers and anganwadi workers (AWW). These two 
groups belonged to different sectors — field-staff are de- 
ployed by the public health department and the AWW, the 
pre-school educator and nutrition worker belonged to the 
Integrated Child Development Program. Field-staff fostered 
acceptability to vaccination through their regular house 
visits and constant interactions with mothers, and AW Ws 
increased the community's awareness of immunisation pro- 
grammes. The vaccine literacy of the households was 
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increased not only by the constant interactions of these two 
workers with the households, but also because of their sta- 
tus in the community as a trusted source of health 
information. 

Within a household, the mothers played a significant 
role in vaccination decisions as she held most of the 
vaccination-related information. The media contributed by 
carrying positive messages about immunization to commu- 
nity households. In the CLD depicting the reinforcing loop 
displayed in Figure 2 we also observed that the prevention 
of diseases through vaccines reinforced the household's ac- 
ceptability to immunization. The widespread acceptability 
of vaccines is reflected by the absence of major opposition 
to immunization programs during this phase, as well as by 
a significant increase in the overall vaccination coverage 
(Kerala = 84%; national average = 54.2%) [18]. 

In a push for polio eradication, a polio vaccination cam- 
paign was introduced in 1995 as part of the Global Polio 
Eradication Initiative [19] by administering additional oral 
polio vaccine to all children under five on two national 
immunization days. Despite initial opposition from some 
medical professionals, the campaign was generally well re- 
ceived, benefiting from ongoing civil society participation. 
The opposition was from some physicians in academic set- 
tings as well as some private doctors who questioned the 
need for additional oral polio vaccine for a state like Kerala, 
when the state already had high routine immunization 
coverage. Initial opposition was neutralized by the public’s 
confidence in vaccination in general and considerable state 
support for the program. For example, according to our re- 
spondents, there was extensive participation in the program 
implementation by several actors, including several govern- 
ment departments, in addition to health, as well as Pan- 
chayats (village level elected governance institution), NGOs, 
and schools. In 2000, a case of polio was reported in Kerala. 
Although this event was seen as a failure of the public sys- 
tem, the impact on the immunization program was not im- 
mediately evident and the efficacy or safety of the vaccine 
was not questioned. 


Phase 2: Opposition 
The polio case in Kerala was followed by a series of crit- 
ical events related to immunization-created feedback 
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Figure 2 Causal loop diagram representing the phase of high vaccine acceptance and coverage. 
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loops that influenced sudden changes to the social ac- 
ceptability of vaccines. Figure 3 displays the CLD that 
shows increased complexity and the new feedback loops 
that emerged. In this phase, which illustrates a number of 
events from 1995 to the present, we note many more actors 
and unexpected consequences — some arising with a delay — 
as well as the emergence of opposition to immunization. 

Though opposition to the polio campaign emerged right 
at the start of the programme, these discussions mostly 
remained within academic circles. In 1999, the federal 
government decided to strengthen the polio eradication 
drive and introduced Intensified Pulse Polio Immunisation 
(IPPI). This included additional polio vaccination days 
and initiated mop-up rounds, which mandated the field 
workers and volunteers to reach out to unvaccinated chil- 
dren in their households and vaccinate them. The imple- 
mentation of the programme was closely monitored by 
the public health authorities for providing logistics sup- 
port and for achieving complete coverage. 

However, some groups began seeing the IPPI programme, 
for which the state had mobilized significant resources, 
as an opportunity to bring out their own grievances. In 
2002, the Kerala Government Medical Doctors’ Association 
(KGMOA) publically questioned the need for IPPI in 
Kerala and referred to the arguments raised by some 
medical professionals in the beginning of the program; 
they announced their non-cooperation with the program. 
The announcement coincided with a strike called by the 
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association for better service conditions and was partly 
used as a bargain for their negotiations. They justified 
their stand in a press conference as a scientific argument. 
Though KGMOA had later backtracked and cooperated, 
this incident had triggered the first open debate on any 
immunization program in Kerala and emboldened many 
other groups, such as alternate system providers and some 
religious leaders, to raise objections against immunization 
programs (refer to reinforcing loop R1). 

The public protest carried out against the IPPI cam- 
paign by the alternative medicine proponents was in fact 
a debate on the superiority of alternative medicine. For 
example, homeopathy professionals, on several occasions 
in the past, had direct confrontation with professionals 
representing allopathic medicine in Kerala. One of such 
conflicts started as far back as the 1970s, when allopathic 
providers opposed the initiation of a graduate programme 
in homeopathy in Kerala — described as the first of its kind 
anywhere in world. A leader of one of the homeopathic 
associations which has strong membership in northern 
Kerala described their campaign against IPP] programme 
as the payback time for the humiliation they had suffered 
from the hands of allopath. 


“We cannot accept it. They declare themselves that 
they are big people; but we don’t feel so. If all the three 
systems need to coexist, then there has to be mutual 
recognition. When we recognise allopathy as a medical 
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system, and if they do not reciprocate it, where is the 
question of dialogue? That is why we not only practice 
[oppose vaccinations], but we also preach to our 
patients against polio |vaccine}.” — Homeopathic 
association leader Khozhikhode 


The alternative medicine providers staged open protests 
in the north, including in the Kozhikode district. A popu- 
lar health magazine of a naturopathy group had carried 
several articles against pulse polio immunisation, one of 
which was by a well-known naturopath who spoke at sev- 
eral public meetings against vaccines, especially the polio 
campaign. From 2005 onwards, opposition to IPPI from 
groups who opposed vaccination in general, such as hom- 
eopathy associations and experts of naturopathy, were 
joined by some religious organizations and was widely 
publicised. They often cited the opposition of IPPI by the 
allopathic professionals and KGMOA. This reinforced 
their arguments and gave credibility for their public pro- 
test (see reinforcing loop R2). Though the same groups 
broadcast their opposition messages across the entire 
state, coverage in the south was not affected by the outcry 
against immunization, but the northern districts began 
to show a decline. In the context of declining acceptabil- 
ity of the polio campaign, the debates that challenged the 
immunization programs received further credibility (refer 
to R3). The change in acceptability, in turn, increased 
public debate; especially as the negative media coverage of 
these events increased, thus causing a further dampening 
effect on vaccine acceptability (refer to balancing loop B1). 

Although these frequent debates were centred on IPPI, 
they began to influence the community's trust in vaccines 
(reinforcing loop R4). Additionally, as the incidence of 
vaccine-preventable diseases was drastically reduced with 
time, the general population felt gradual loss of fear due 
to relative unfamiliarity of vaccine-preventable diseases 
in the state. This subsequently reduced routine vaccine 
acceptability and created a dampening effect on vaccine 
acceptability (see balancing loop B2). The reinforcing loop 
R5 denotes a potential amplifying effect of low levels of 
vaccine acceptability over a long period of time on a 
household’s trust in vaccination. In the face of losing inter- 
est in immunization, the health workers found it difficult 
to convince the parents to vaccinate their children. 


“Earlier we had cases to show to people, now they are 
not seeing cases; it is now like a riddle to them. We 
now feel that the days ahead will be even tougher” — 
Health worker (female), Kozhikode 


In 2006, a death was reported after a school immunization 
program in Kozhikode district. As a result, there was an 
eruption of immediate public protest and violence against 
the local public health staff and facilities, as the safety 
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of vaccines were again challenged [20,21]. Immunisation 
programmes, and especially the field immunisations and 
school-based immunisation programmes, had to be stopped 
in most districts of northern Kerala. 

Table 1 summarizes the critical events described above, 
and, in retrospect, their impact on the immunization 
system. 

Though the routine immunisation programme was car- 
ried out in a similar manner, and both districts initially 
reached very high coverage levels, the northern district 
ended up with a decline in coverage for routine vaccina- 
tions. Protest against IPPI by the practitioners of alternate 
medicine, especially homeopathy, which is more popular 
in northern Kerala, was a major factor. In northern Kerala 
many homeopathic practitioners have actively discour- 
aged their clients from immunising their children. 
Several study respondents believed that the strong in- 
fluence of homeopathic medicine practitioners on house- 
holds in northern Kerala helped convincing them against 
immunisation. 


“Because they |homeopathy practitioner] have a 
family physician status and have good relationship 
with some of their clients, they oppose them from 
accessing the mainstream |allopathic] system. This is 
the limitation of the health department personnel. We 
are not fully able to reach to them.” — Health official 
(male), Kozhikode district 


In northern Kerala, the popular synonym for any vac- 
cination has always been ‘polio injection’. This pointed 
to the possibility that a targeted campaign against polio 
immunization in northern Kerala perhaps had an impact 
larger than its objective of opposing repeated polio drops. 
One of the experts interviewed reasoned that as the soci- 
ety did not differentiate between polio vaccines and the 
other vaccines, the resistance against immunization, which 
originally initiated against polio campaign, might have 
moved to routine immunization. 

Even though the routine vaccination was well accepted 
in the southern district, a school-based program for 
Japanese encephalitis introduced in Alappuzha district 
by the public health department created wide-spread 
public debate on the rationale and safety of vaccines. 
The limited success in the Japanese encephalitis program 
was explained by multiple media reports on increasing 
the side effects of the vaccine, challenges of the health 
department in collaborating with resistant schools, and 
parents refusing to send their children to school on cam- 
paign days. Though there was no immediate impact of the 
events on routine immunization, experts, interviewed for 
this study, have suggested future potential negative feed- 
back (represented by a doted arrow in Figure 3) of such 
debates on routine vaccination. 
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Table 1 Major events and its influence on immunization coverage 


Landmark events Period Characteristics of the events — | ct on" 7 

EPI . re 1978 State supported immunisation programme, lower coverage Low vaccination coverage. 
due to lesser vaccine acceptability and supply constraints. 

Introduction of UIP 1985 Strong political commitment to immunisation, extensive Improvement in vaccine acceptability. 
organisation for the delivery of immunisation service, Overall coverage began to improve. 
weekly immunisation. 

Introduction of polio 1995 Introduced as special event and additional vaccines in addition Improved coverage for routine immunisation. 

eradication initiative to routine. Polio eradication drive was well received by Successful implementation of polio 
community. Active participation of civil society in its eradication drive. 
implementation. Doubts against oral polio vaccines raised 
by some from public health community. 

Last case of polio 2000 Widely reported in Kerala. This was seen as a failure of No specific impact on immunisation 

reported in Kerala public health system. programme. 

Opposition of IPPI 2002 The opposition coincided with the strike call by the association. | No immediate impact on vaccination 

immunisation strategy The issues were widely discussed in the media. The association coverage, however, first open debate in 

by KGMOA backtracked after the strike was resolved. Kerala on any immunisation programme. 

Organised opposition 2005 Opposition was mainly from homeopathy groups and some Immunisation coverage in northern districts 

against from IPP! by onwards experts of naturopathy, also supported by some religious showed a decline. Coverage is intact in 

various groups organisations. Motive of continuing IPPI was openly southern districts. 
| i i le. 
discussed from both a technical and conspiracy angle Increased feB@nig erm eases tn 
northern Kerala. 

Death reported after 2006 Eruption of immediate public protest and violence against Reported to have had an impact on 

school vaccination the local public health staff and facilities. Widespread anguish vaccination coverage in northern districts. 

programme in a felt among public as the safety of vaccines were challenged. 

northern district 

Special school based 2008 Wider public debate, on the rationality and safety of vaccination Limited success in Japanese encephalitis 


Japanese encephalitis 
vaccination programme 
in Kerala 
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Impact on immunisation 


programme. Media reports of vaccine side-effects impacted the 
programme. At the implementation level, difficulties in 
organising events in collaboration with schools. 


vaccination programme. 


EPI, Expanded Programme for Immunisation; IPPI, Intensified Pulse Polio Immunisation; KGMOA, Kerala Government Medical Doctors’ Association; UIP, Universal 


Immunization Programme; VPD, Vaccine Preventable Disease. 


Official response to decreasing immunization coverage 
An immediate state public health department's response 
to the decline in the public image of immunization in 
northern Kerala has been to strengthen the programme 
which included assigning high immunization coverage 
targets to staff at all levels and a close scrutiny of their 
achievements. It was observed, for example, that during 
the regional review meetings, if the coverage of a par- 
ticular vaccine was less than expected, a close scrutiny 
and justification was sought from the fieldworker and 
the supervisor. 

Over-emphasis on coverage targets created perverse 
incentives for health providers to inflate their coverage 
estimates, and has made the immunization coverage data 
generated by the health department grossly unreliable as 
evident from several independent surveys which re- 
ported significantly lower levels of coverage. Besides, it 
often resulted in coercion of resistant households for 
vaccination. During the mop rounds for polio eradica- 
tion campaign, heated exchanges between health field 
workers and members of households that are resistant 
to immunize their children were observed in many 
places; “...may be because when it is forced, they may 
think it is for the others benefit not for their benefit” — 
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commented one of the experts who is also a district level 
immunization program manager. 

Another strategy used in responding to the crisis was 
to confront with the groups that opposed immunization. 
For example, the public health administration, which is 
dominated by allopathic system of medicine, retaliated 
against homeopathic professionals by issuing a govern- 
ment order to set-up vaccine booths in government- 
owned homeopathy dispensaries. They described this as 
a strategic move and prominently projected it in a press 
conference and in the press statements issued on the 
previous day of the campaign. The homeopathy practi- 
tioners’ association found this step as intimidating and 
one of their office-bearers who was interviewed commu- 
nicated the association’s resolve to oppose the program 
more strongly. 

The official response also included the use of commu- 
nity intermediaries to counter the misinformation against 
immunization. However, many of the field staff in Kerala 
had accepted that their ability to influence families in 
health-related decisions have reduced over the years, espe- 
cially with the decline in the frequency of house visits. 
The introduction of community health workers known as 
Accredited Social Health Activists (ASHAs) has, therefore, 
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played a positive influence in decision making of the par- 
ents on immunization. ASHA’s status as local women 
known to the other members of the community gave her 
special advantage in influencing the perceptions of com- 
munity on immunization issues. During her house visits 
information is shared as part of a day-to-day communica- 
tions. The messages related to health and immunization 
gets discussed and exchanged during such interactions. In 
addition, in their role as teacher of pre-school children, 
AWWs also had special access to mothers when they 
come to drop or pick up their children from the pre- 
school and used the opportunity to influence mothers’ be- 
haviour towards immunizing their children. One of the 
experts interviewed had noted that the health department 
overlooked the potential of AWWs which resulted in 
gradual decline of the role of AWWs in immunization. 
The future potential of the right kind of interactions of 
AWWs and ASHAs with the households to create reinfor- 
cing loops to improve trust in vaccination is denoted by 
dotted arrows and loops in the Figure 3. Likewise, we also 
anticipate a change in disease situation due to reduction 
in vaccine coverage which may increase vaccine accept- 
ability in future and improve household’s trust in vaccines. 

Nevertheless, an important limitation of the strategy 
to use community level workers, such as ASHAs, to im- 
prove household acceptability of immunization was 
often observed during home visits. The mothers whose 
children were not vaccinated said that the decision not 
to vaccinate was taken by the male members of their 
households. “When we speak to mothers many of them 
will point fingers at husband, father-in-law, or mother- 
in-law.” — commented one of the paediatricians inter- 
viewed when asked about her ability to convince families 
who refuse vaccination. There was a greater likelihood 
that the male members got influenced by external fac- 
tors, such as the media and the public protests, used by 
the alternative medicines groups. Given that the 
mobilization strategies used by the public health system, 
including ASHAs, were often designed to target mothers 
in the households and community, the impact of these 
strategies was not as intended given the important role 
of the male members. 


Discussion 

Using a CAS lens facilitated the identification and under- 
standing of unintended consequences and unexpected 
phenomena. Our CLDs illustrated the complexity under- 
lying immunization coverage in the northern districts of 
Kerala and showed that the campaigns and the messages 
targeting against special immunization programs by some 
of the interest groups had consequences larger than 
intended as it affected households’ acceptability of routine 
immunization. The events that occurred at different 
points in time had a delayed and cumulative impact on 
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vaccine acceptability. Our findings also showed that sev- 
eral informal day-to-day societal interactions within the 
households and community played crucial roles in creat- 
ing and changing vaccine acceptability. For example, the 
decision making for immunization during the phase of 
vaccine resistance in northern Kerala showed a prominent 
role of male members of the households in contrast to the 
role played by mothers during the acceptability phase. 

A crucial question proposed is regarding how the pub- 
lic health departments governing immunization programs 
can retain or regain vaccine acceptability in complex con- 
texts such as in Kerala. Our CLDs shows that several 
events, related or unrelated to immunization programs, 
affected vaccine acceptability through new actors and their 
interactions (Figure 3). These new interactions influenced 
the pathways of feedback that created vaccine acceptability 
in the beginning (Figure 2). The comparison of two re- 
gions in Kerala showed greater impact of feedback from 
actors, such as practitioners of alternate systems of medi- 
cine, negatively affecting vaccine acceptability in northern 
Kerala as they have stronger influence over households. 
Therefore, understanding what factors influenced the 
direction of the feedback and modulated its potential 
to impact the vaccine acceptability pathway assumes 
importance. 

The study findings, as well as our review of the literature, 
shows ‘trust’ as an important factor that modulates this 
feedback between actors [22,23]. For example, our findings 
reveal that from a period of suspicion and rejection, vac- 
cines have achieved public confidence mainly through a 
positive feedback mechanism facilitated by its capacity 
to demonstrate reduction of diseases in the community 
(Figure 2). Information against immunization programs 
and the occasional reporting of vaccine-related adverse 
events undermined a household's trust in immunizations. 
In the context of a low burden of vaccine-preventable 
diseases (due to several years of good vaccination cover- 
age), the reductions in trust created a negative feedback 
loop that dramatically affected vaccination acceptability 
and coverage. We discuss our findings in the light of two 
theoretical interpretations of trust; trust in expert sys- 
tems, and interpersonal trust, to understand the feed- 
backs and to explore strategies for the better governance 
of immunization services [22]. 

In the first view, acceptability to immunization in the 
initial phase in Kerala can be viewed as a result of the 
trust in institutions of professional expertise (in this 
case, medical knowledge) [24]. However, the conflicting 
messages that emerge from different systems of medi- 
cine challenge the trust which people attribute to expert 
systems of vaccination. The studies on vaccine resistance 
in other contexts, too, have noted sudden loss of trust in 
vaccines when counter campaigns question the scientific 
basis of vaccinations [25,26]. Consistent health messages, 
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especially from different sources of expertise, is import- 
ant and can be achieved by engaging in dialogue that 
generates consensus rather than direct confrontation 
with other systems of medicine, as attempted by the 
public health department in Kerala. We also see a possi- 
bility of immunization programs regaining household 
trust when vaccine-preventable diseases reappear with 
the decreasing coverage (denoted by a dotted arrow with 
delay mark in Figure 3). 

The role that the media played in such contexts in 
informing households with conflicting messages on 
immunization and how the key actors trusted each infor- 
mation requires attention. As observed by Giddons, 
plurality of information, a feature of late modernity, is a 
reality of many societies in the developing world and has 
implications on governing public health functions like 
immunization services [27]. For example, in Kerala, where 
increased penetration of 24 hours local electronic news 
media and several widely-read health publications, are 
informing households on every immunization-related 
side-effect and public debate on immunization programs. 
Sensitizing the media for more responsible reporting and 
using it to convey appropriate health messages are options 
that public health departments may use in such situations, 
even though it is unlikely to eliminate all unwanted infor- 
mation from reaching households. 

In the second interpretation, trust is approached as a 
cognitive phenomenon or a judgment based on a ra- 
tional gamble and therefore household perception of 
other actors’ interests is important. One of the strategies 
of the public health department to influence the house- 
hold level decision making using community level func- 
tionaries like ASHAs, therefore, has a potential (denoted 
by a dotted arrow in the Figure 3). Because of the social 
networks in which ASHAs are interconnected, house- 
holds can perceive them to serve their interests. Previous 
studies from other health system settings also conform 
this finding [28]. A study of treatment-seeking behaviour 
in urban Sri Lanka noted the perception of community 
on the motives of healthcare workers as a central factor 
to the formation of trust in health services, especially in 
the face of uncertainty about health conditions [29]. 

Trust in health workers can also be explained through 
the notions of ‘affective trust? which is developed through 
emotional bonds and obligation generated through their 
repeated personal interactions with the households [28]. 
However, we observe that the trust created by community 
level health functionaries is intrinsically related to how 
they interacted with the households. In the backdrop 
of widespread misinformation against immunization in 
Kerala, when the functionaries are forced to deliver against 
rigid performance targets, it leads to coercive practices 
and may undermine the health worker's long-term trust 
with the community. Thiede’s analysis of origin of trust 
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and mistrust in healthcare draws a similar conclusion that, 
while trust enhanced communication of health workers, it 
was the process of communicative interaction that gener- 
ated trust in the first place [23]. 

In the bureaucratic context of implementation of UIP, 
there was limited recognition of the need to influence 
informal interactions that retains trust in vaccines and in 
the public health department that governs immunization 
programs. The governance of immunization was seen as 
an exercise to ensure control and order through top-down 
hierarchical interactions. At each level, the program is 
conceived as an array of demands to be met. This absolves 
the capacity of the system to adapt to emerging complex- 
ity. Similar observations on public health bureaucracy 
have been made by other studies that looked at target- 
driven top-down implementation of public health pro- 
grams in the South-East Asian context [30-32]. 

This study has various strengths and limitations. 
Using a CAS framework to guide data analysis and 
interpretation contributed to understanding the com- 
plexity involved in the governance of immunization 
services in a developing country context. It showed 
how system thinking concepts and methods can be 
applied to a complex question such as changing house- 
hold acceptability to immunization. We also showed 
how tools such as CLDs can be used to explore social 
phenomena interlinked to governance of public health 
functions and interpret feedback loops that influenced 
the change in vaccination coverage. Development of 
CLDs based on content analysis of qualitative data and 
using these CLDs to guide further thinking of the com- 
plex system behaviour is arguably a unique feature of 
this analysis. 

There are, however, several limitations to this study. 
Firstly, it included only two districts in the analysis of 
the reasons for the change in vaccination coverage in 
Kerala and it may not necessarily apply to other settings. 
Though the resistance against immunization is wide- 
spread in Kozhikode district, the intensity of resistance 
is varied in different regions of the district. Our descrip- 
tion of resistance phenomenon is only pertaining to the 
areas that showed significant levels of resistance and 
cannot be applied to regions with good vaccination ac- 
ceptance. This study tries to interpret and propose 
corrective strategies only on the basis of an ex-post 
analysis of complexity and therefore does not reflect 
on the experience of a health system that regained 
vaccine acceptability after losing it. For example, the 
study did not reflect on the changes in vaccination cover- 
age in Alappuzha district that had occurred in the earlier 
phase. The CLDs could not be validated, and therefore 
may reflect inaccurate linkages. Additionally, it is 
possible that the authors omitted potentially import- 
ant variables and events. 
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Conclusions 

We argue, through this article, that the evidence base of 
public health programs, such as immunizations, should 
go beyond epidemiological and economic analysis. Our 
study emphasizes the need for public health governance 
systems to take into consideration the nature of multiple 
interactions when societies organize themselves to man- 
age a public function like immunization. This perspec- 
tive goes beyond the conventional assumption that the 
government's public health department is the sole gov- 
ernor of public health issues divorced from wider soci- 
etal forces such as other key providers, social networks, 
and the households themselves with which the decision 
to vaccinate lie. It should make a careful consideration 
of multiple interactions involving the actors and their 
perceptions and ideas, which are shaped by factors such 
as trust. 

The traditional approach to public health governance 
is directed by hierarchical organization designed to direct, 
control, and/or to even prevent interactions. However, 
complexity ensures that interactions will lead to unpre- 
dictable changes. The effort of the public health depart- 
ment should be to influence the multiple interactions of 
various governance actors and institutions. 
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Abstract 


Background: Assuring equitable universal access to essential health services without exposure to undue financial 
hardship requires adequate resource mobilization, efficient use of resources, and attention to quality and 
responsiveness of services. The way providers are paid is a critical part of this process because it can create 
incentives and patterns of behaviour related to supply. The objective of this work was to describe provider 
behaviour related to supply of health services to insured clients in Ghana and the influence of provider payment 
methods on incentives and behaviour. 


Methods: A mixed methods study involving grey and published literature reviews, as well as health management 
information system and primary data collection and analysis was used. Primary data collection involved in-depth 
interviews, observations of time spent obtaining service, prescription analysis, and exit interviews with clients. 
Qualitative data was analysed manually to draw out themes, commonalities, and contrasts. Quantitative data was 
analysed in Excel and Stata. Causal loop and cause tree diagrams were used to develop a qualitative explanatory 
model of provider supply incentives and behaviour related to payment method in context. 


Results: There are multiple provider payment methods in the Ghanaian health system. National Health Insurance 
provider payment methods are the most recent additions. At the time of the study, the methods used nationwide 
were the Ghana Diagnostic Related Groupings payment for services and an itemized and standardized fee schedule 
for medicines. The influence of provider payment method on supply behaviour was sometimes intuitive and 
sometimes counter intuitive. It appeared to be related to context and the interaction of the methods with context 
and each other rather than linearly to any given method. 


Conclusions: As countries work towards Universal Health Coverage, there is a need to holistically design, 
implement, and manage provider payment methods reforms from systems rather than linear perspectives, since the 
latter fail to recognize the effects of context and the between-methods and context interactions in producing net 


effects. 
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Introduction . 
In 2005, the member states of the World Health Organi- 


zation committed to Universal Health Coverage (UHC) 
[1]. Specifically, their commitment was to develop their 
health financing systems so that their citizens would 
have universal access to essential health services (defined 
in context) without having to suffer financial hardship in 
paying for them. Subsequently, in 2012, the United 
Nations General Assembly, in resolution A/67/L.36 of it 
is 67th session [2], called upon member states “to value 
the contribution of Universal Health Coverage to achiev- 
ing all inter related Millennium Development Goals with 
the ultimate outcome of health improvements...”. For the 
purposes of this paper, we use UHC in our understand- 
ing of the essence of the World Health Organization 
definition of ensuring equitable universal access to a 
core package of essential health services without expos- 
ing people to undue financial hardship [3]. The details of 
this ideal will have to be defined in context; in all con- 
texts it requires adequate resource mobilization as well 
as the equitable and efficient use of available resources. 
A critical part of this effort will be provider payment 
methods [4]. 

Provider payment methods refer to the mechanisms 
used to transfer funds from the purchaser to the providers 
of health services. These methods include line itern and 
global budgets, salaries, capitation with or without fund 
holding for referral services, case-based payments, and fee 
for service with or without a fee schedule. The provider 
payment system, on the other hand, refers to the payment 
method combined with all supporting systems such as 
accountability mechanisms, management information sys- 
tems, etc. Different provider payment methods create dif- 
ferent provider behavioural incentives related to service 
supply since they have different effects on the relationship 
between provider income and costs for providing the ser- 
vice, as well as the relationship between activities and 
payment [4,5]. Apart from financial incentives, provider 
supply behaviour can be influenced by other factors, such 
as peer, professional, and client pressure, and factors in- 
ternal to the provider such as value systems and ethics. 

Also important to understanding incentives is that 
provider payment methods are introduced and imple- 
mented in health systems. Systems are made of separate 
but interdependent parts that interact with each other. 
Occurrences and outcomes within systems can only be 
fully understood by appreciating the relationship and 
interconnectedness between these parts [6-8]. Moreover, 
health systems are complex adaptive systems (CAS), con- 
stantly changing and governed by feedback. Intervening in 
one part of the system will almost always have ripple 
effects in other parts; they self-organize and adapt based 
on experience. To fully understand incentives in a CAS; it 
is important to apply a systems thinking perspective, 


Page 2 of 17 


studying the context in which the payment method has 
been introduced and the resulting interactions. 

The current study therefore set out to explore, from a 
systems thinking perspective, the questions of: “What 
kinds of provider behaviour are occurring related to sup- 
ply of health services to insured clients in Ghana?; What 
incentives might be driving the behaviour?; and What is 
the influence of provider payment methods on the in- 
centives and the behaviour?” Our focus was on financial 
incentives for service supply behaviour related to the na- 
tionwide National Health Insurance (NHIS) provider pay- 
ment methods of the Ghana Diagnostic Related Groupings 
(G-DRG) for services and itemized fees with a fee schedule 
for medicines. A per capita payment for primary care, 
which was an early pilot in one region at the time of the in- 
vestigation, was not included in our research given the 
focus on nationwide payment methods. The focus on fi- 
nancial incentives was selected since behaviour motivated 
by financial incentives (real or perceived) was and remains 
a source of much debate, conflict, and concern within the 
Ghana NHIS and links closely with concerns about cost es- 
calation and cost containment. 


Context: economic, socio-demographic, and health 

After a long period of near stagnation, Ghana has seen 
rapid growth in its GNI from an estimated US$ 320 per 
capita in 2003, when the NHIS law was passed, to US$ 
1,410 (Atlas method current US$) in 2011 [9]. It is trad- 
itionally an agricultural country with cocoa, timber, and 
gold as its main exports. Oil was discovered off-shore in 
2006 and production in commercial quantities started in 
2011. The amounts produced are still small, but the im- 
portance of oil to its economy is growing, and it has 
played some role in the evolution of its GNI per capita. 
The Consumer Price Index, which measures the per- 
centage change over time in the general price level of 
goods and services in a country, has risen each year and 
has remained high over several decades. Annual averages 
since 2003, when the NHIS was established, have ranged 
between 10% to 27% [10,11] and the value of the cedi 
has declined against the dollar. 

About half of Ghana’s estimated population of 26 mil- 
lion are below 15 years old. The majority of formal sec- 
tor workers, with some exceptions, such as employees 
of some tertiary educational institutions, belong to the 
Social Security and National Insurance Trust (SSNIT) 
pension scheme. Based on the 2011 SSNIT annual re- 
port, 963,619 Ghanaians (about 4% of the total popula- 
tion) were active contributors [12]. Even if the figures 
are doubled to include formal sector workers who do 
not contribute to the SSNIT pension scheme, it would 
be reasonable to estimate that about 80% of Ghana’s 


adult working population is employed in the non-formal 
sector. 
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Mortality of children under 5 years has declined, albeit 
very slowly, from 155 per 1,000 live births in 1983 to 
1987, to 80 per 1,000 live births in 2003 to 2008 [13]. 
Maternal mortality declined from 503/100,000 in 2005 
to 451/100,000 in 2008 [14]. Shortages of skilled human 
resources have been and remain a problem. The World 
Health Report 2006 estimated that Ghana had 0.15 physi- 
cians and 0.92 nurses per 1,000 population. This com- 
pared with 2.14 and 9.95 in a high-income country like 
Canada and 0.77 and 4.08 in a sub-Saharan Africa middle- 
income economy like South Africa [15]. The country’s 
challenges, with inadequacies in infrastructure, equip- 
ment, tools, and supplies in the health sector, mirrors 
its human resource challenges. A little under 15% of the 
public sector budget is allocated to health and the per 
capita expenditure on health in 2013 was estimated at 
US$35 [16]. 

The Ghana Health Service, the service delivery agency 
of the Ministry of Health, employs most public sector 
providers. Others are employed by other public sector 
agencies with hospitals of their own, e.g., the Military, 
Police, and the Universities. Private service delivery is 
done by not-for-profit and self-financing (for-profit) 
providers. Mission clinics and hospitals under the um- 
brella of the Christian Health Association of Ghana 
(CHAG) form most private not-for-profit providers. 
The private self-financing sector is made of individual 
physician, dentist, and midwife practices, hospitals, la- 
boratories, and pharmacies. 


The Ghana National Health Insurance Scheme (NHIS) 

In September 2003, Ghana passed a national health insur- 
ance (NHI) law (Act 650) to replace public sector user fees 
introduced in the 1980s as part of structural adjustment 
programs. Though the term UHC was not used, the gov- 
ernment’s stated policy objectives in setting up a NHI 
scheme show the principles of UHC. Both the original 
[17] and revised [18] NHI policy frameworks state: Euiste 
vision of government in instituting a health insurance 
scheme.... is to assure equitable and universal access for 
all residents of Ghana to an acceptable quality package of 
essential healthcare... every resident of Ghana shall belong 
to a health insurance scheme that adequately covers him 
or her against the need to pay out of pocket at the point of 
service use in order to obtain access...”. 

The Ghana NHIS is described in several publications 
[19-22]. The benefit package covers about 80% to 90% of 
the most common clinical conditions in Ghana. The 
NHI has a single payer arrangement through the NHI 
fund. The NHI fund is about 70% to 75% from a value 
added tax and 20% to 25% from formal sector SSNIT 
contributions, 2.5% of which are mandated to be trans- 
ferred into the NHI fund monthly. A small amount of 
NHI financing comes from the annual premium, non- 
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SSNIT contributors pay out of pocket and the registra- 
tion fee paid by all subscribers. 


Nationwide NHIS provider payment methods 

Ghana’s NHIS started implementation in 2004, with 
itemized billing with no standardized fee schedule for 
services and medicines as the provider payment method 
for public and private service providers. Each of the dis- 
trict schemes negotiated with their providers itemized 
fee rates for services, consumables, and medicines. In 
the face of growing concerns over inefficiencies such as 
random price variations for the same procedures and con- 
sumables, cumbersome billing and claim vetting proce- 
dures and cost escalation, the National Health Insurance 
Authority (NHIA) introduced, in 2008, a case based pay- 
ment system known as the G-DRG for services and proce- 
dures, as well as standardized itemized fees for medicines 
based on a medicine list. Apart from a few modifications, 
this payment system has remained in use across Ghana in 
its original design since then. 

Classically, the two core components of a DRG pay- 
ment system are a patient classification system and a 
payment rate setting mechanism that takes into account 
the intensity of resources used to treat patients in a 
given DRG category to give cost weights or prices to the 
DRG [23]. The G-DRG is not a pure DRG system in 
that, although it has the patient classification system, it 
does not have cost weights and severity levels. It was de- 
signed, applied, and continues to be applied nationwide 
for all levels of care from the lowest (Community Health 
Planning and Services (CHPS) compounds) to the high- 
est (teaching hospitals), to pay all accredited providers — 
public, quasi-government, and private — for inpatient 
and outpatient services. The tariffs reflect preceding 
charges rather than a precise or economic costing; cap- 
ital and equipment costs are not included. The tariffs are 
classified into three broad groups of diagnoses, proce- 
dures/operations, and investigations. The calculated dir- 
ect cost of the services for consumables and labour are 
uniform for related or similar diagnosis, procedures, and 
investigations irrespective of level of care. 

Indirect or overhead costs comprising labour, vehicle 
maintenance and fuel, equipment and building mainten- 
ance, housekeeping, utilities, and general administrative 
and office expenses are calculated, increasing from the 
lower to the higher level of care. The rationale is that facil- 
ities at higher levels of care consume larger amounts of 
overhead inputs because of their size and higher fragmen- 
tation of services. The tariffs vary according to whether 
the facility is government, mission, or private to take into 
account the government subsidy, mainly for salaries but 
also some infrastructure, equipment, and overhead costs 
in the public and, to some extent, the private mission sec- 
tor, as well as the zero subsidy in the private self-financing 
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sector. The tariffs also vary by type of final service 
(inpatient or outpatient), type of intermediate service 
(laboratory investigations, imaging investigations, theatre, 
catering services), and specialty (obstetrics and gynaecol- 
ogy, medicine, surgery, child health, eye, ENT, and dental). 
Since some district hospitals have catering services and 
others do not, inpatient tariffs differ by district hospitals 
with catering services and those without [24]. 

The itemized fee schedule for medicines is based on a 
NHI medicines list (NHIML) that is periodically revised. 
Medicines can be dispensed by public and private pro- 
vider facilities with an in house pharmacy/dispensary or 
by private community practice pharmacies accredited by 
the NHIA. Most community practice pharmacies, like 
other private self-financing services, are based in wealth- 
ier and peri-urban areas. Poorer rural communities rely 
on chemical sellers (lay people licenced by the Pharmacy 
Council to sell over the counter medicines). Some of 
these are also accredited by the NHIA. In theory, there 
is supposed to be a separation between prescribing and 
dispensing; in practice, it is not enforced. 

Payment to providers for services and medicines was 
and remains retrospective. Providers file claims, which 
go through a vetting process in the NHIA district scheme 
offices or for the higher-level facilities such as teaching 
and regional hospitals in the computerized central claims 
processing office of the NHIA, before final payment. The 
claims processes of many provider and district scheme of- 
fices remain predominantly manual despite increasing 
computerization. There remain administrative capacity, 
human resource, technical, and other challenges that slow 
down the process and can reduce the final value of the re- 
imbursements [25]. 

The first review of the G-DRG tariffs after introduc- 
tion in 2008 occurred when the Minister for Health, in 
response to provider agitations, announced an interim 
upward adjustment of G-DRG tariffs effective 1 July 
2011. The increments were calculated based on an ana- 
lysis of the trends in medical inflation since 2008, when 
the G-DRG was first introduced. Inpatient tariffs were 
increased by 30%, primary outpatient care by 22%, diag- 
nostic services by 22%, and secondary and tertiary out- 
patient care by 25%. In that same year (2011), the first 
formal review of the G-DRG was commissioned by the 
NHIA. Stallion & Milliman Consultants were engaged 
for this review whose objectives were to: “simplify the fee 
system, increase transparency and ensure that the G- 
DRG developed were consistent with Ghana’s standards 
of treatment” [26]. The review was completed in 2012 
and resulted in a further upward adjustment in the rates 
for all the G-DRG and some changes in the G-DRG 
groupings with some removed or merged, or new ones 
developed. The overall average change in the G-DRG 
tariffs was about 26% above the rates that were set in 
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July 2011. Implementation of the new tariffs started on 
Ist February 2013. 

The first review of the NHIML and prices was in 
October 2009, the second in March 2011, and the third 
in July 2013. Data on the percentage increase in tariffs 
for the first and second review could not be found. 
However, for July 2013, the rise in tariffs was about 12% 
above the preceding levels. 

Many inputs into health service delivery in Ghana are 
imported. Figure 1 shows the total value of NHIS reim- 
bursements for medicines and services to providers over 
time. In cedi terms, the amounts have risen steeply, in 
dollar terms (exchange rates at 4.00 pm UT on 30th 
June each year) the rise is slower and flattening. 


Other provider payment methods 

The health system had other provider payment methods 
with which the G-DRG for services and itemized fee 
schedule for medicines came to co-exist. The Govern- 
ment of Ghana line item, global budgets, and salary pay- 
ments provide supply side subsidies to public providers 
from consolidated tax funds for service delivery, admin- 
istration, infrastructure, equipment, tools, and supplies. 
Some supply side subsides to public sector providers 
also come from donor Sector Budget Support and pro- 
gram funding. Allocation of Government of Ghana funds 
to public sector facilities is often based on historical 
budgets despite the theory that with the Medium Term 
Expenditure Framework reform these budgets would to 
be tied to Ministry, Department, and Agency vision, mis- 
sion, objectives, and plans of action for the year. A major 
reason for this would appear to be that the national 
budget is so constrained it makes it difficult to relate al- 
locations to requests. Fund flows also tend to be irregu- 
lar and unpredictable in amount. CHAG facilities also 
receive supply side subsidies since a large proportion of 
staff salaries are paid from Government of Ghana Con- 
solidated Funds. Most CHAG facilities are located in 
underserved areas, considered as priority for service de- 
livery, and are seen as supporting government to attain 
its equity and access goals in service delivery. 

Private self-financing (for-profit) providers receive no 
government supply side subsidies. They rely for their in- 
come on activity-based payments related to services and 
population, namely out of pocket payments by clients, 
direct reimbursements by some corporations, and, since 
2004, NHIS reimbursements. Sometimes these providers 
chose not to participate in the NHIS because they con- 
sider the tariffs inadequate. Before the introduction of the 
NHI scheme clients in public and private sectors paid out 
of pocket fees based on itemized fees with no fee schedule. 
Non-insured clients continue to pay these fees in both 
sectors. Some public sector providers earn extra income 
through part time locums in private facilities. In some 
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instances they may actually own a private practice. Re- Ashanti region of Ghana, which has 19% of the popula- 
ports of under the table charges by some public sector _ tion, since January 2012. Plans to scale up per capita pay- 
providers also exist but it is hard to document the extent ment for primary care nationwide have been announced. 
of the practice. Figure 2 summarizes the purchasers and providers in the 

Implementation of a pilot per capita payment method Ghana health system, method(s) used by each purchaser, 
for primary outpatient care has been on-going in the and fund flow from purchaser to provider. 
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Theoretical framework 

There are several theories of provider behaviour that 
predict incentives and supply responses to payment 
methods. Building on literature reviews [4,5] of provider 
payment methods and the financial incentives for supply 
behaviour they potentially generate we theorized that 
supply can be modified in two broad dimensions. One 
dimension is related to the numbers of client encounters 
a provider has in a given time period. This dimension 
can change by an increase or decrease in the individual 
numbers of clients making up the provider’s client pool, 
or in numbers of visits per client for the same pool, or a 
combination of the two. Question in relation to this 
dimension would be whether there are incentives for 
providers to try to increase or reduce numbers of en- 
counters. A variety of means, such as modifying opening 
and closing hours, referring or not referring clients to 
other providers, making the service more or less attract- 
ive to clients, etc., could be employed by providers to 
affect this dimension. Incentives to have more client 
encounters would not be infinite but bounded by the 
provider infrastructure, equipment, tools, supplies, and 
human resources, as well as the perceived and actual 
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value of the alternative uses of provider’s time and 
resources, 

The second dimension of supply would be related to 
the inputs into the services provided in each client en- 
counter regardless of the number of encounters. The 
manifestation of this dimension would be related to 
incentives to supply more or less medicines, laboratory 
tests, procedures, etc. Again, the incentive to supply 
more or less would be bounded rather than infinite. 
These two dimensions could be simply summarized in 
the form of a graph as in Figure 3A. The theorized ex- 
pected incentives in these two dimensions for each of 
the provider payment methods operational in Ghana, 
based on the review of the literature, without analysing 
the effect of context and interactions with other pro- 
vider payment methods in the system, can be mapped 
onto this graph as in Figure 3B. 

Since our objective was to understand service supply 
behaviour and incentives related to the NHIS provider 
payment methods in the context of Ghana and its health 
system, we went beyond the simple linear theoretical 
model of Figure 3A,B and drew on realistic evaluation 
[27] and systems thinking theories including the concept 
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of the health system as a CAS [6] for our analysis. We 
have already described features of CAS and the rele- 
vance to this study in our introduction. 

Realistic evaluation theory suggests that a particular 
action(s) leads to outcome(s) or effect(s) by triggering a 
mechanism or set of mechanisms acting in context. The 
link between action and outcome is thus complex and 
non-linear. Thus, the observed effects of provider pay- 
ment methods in the Ghanaian health system may not 
manifest as direct linear causative effects between the 
payment method and the observed service supply behav- 
iour as theorized in Figure 3B. Rather, any given pro- 
vider payment method will interact with the context and 
other methods to trigger mechanisms that would lead to 
observed service supply behaviours and incentives. They 
could, but would not necessarily be, as predicted from 
the theoretical linear analysis in Figure 3B. 


Methods 

The study was carried out over a 6-month period from 
March to August 2013 using mixed methods of data col- 
lection. Google and PubMed searches for “provider pay- 
ment methods in Ghana”, “provider payment systems in 
Ghana”, “Ghana DRG payment system”, and “Ghana Na- 
tional Health Insurance Scheme” were used to identify 
grey and published literature for review. The Ministry of 
Health, Ghana Health Service, and NHIA websites were 
searched for reports with relevant information. Addition- 
ally, key informants were asked whether there were any 
reports, administrative memo, and other material in their 
records and archives related to provider payment methods 
that could be made available for review. The search was 
focused on the period January 2003 to August 2013. Rou- 
tine management information system data of providers 
and schemes related to utilization and claims over the 
same period was obtained for secondary analysis. Some of 
the national level provider data could not be obtained for 
the period 2005 to 2007. 

Primary data collection at the regional and district level 
was conducted between April and June 2013. Follow-on 
interviews and two validation meetings with respondents 
to discuss our initial analysis and conclusions were con- 
ducted in July/August 2013. Table 1 summarizes the geo- 
graphic location of primary data collection, methods of 
data collection, and number of respondents for each 
method employed. The study had several questions be- 
yond the ones presented in this paper. We only describe 
variables and indicators from which data was drawn to an- 
swer the questions of this paper. 

The national level key informant interview guide items 
explored how the G-DRG and itemized fee schedule for 
medicines payment methods were designed and imple- 
mented, and perceptions of service supply incentives 
and behaviour related to the design and implementation. 
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At the district level, key informant interviews were held 
with District Insurance Scheme office managers, District 
Health Directorate staff, and health facility managers. 
Areas covered in the interviews were observations and 
perceptions of how the NHIS provider payment system 
had affected health facility and insurance scheme office 
decisions related to service supply and the advantages 
and disadvantages of the methods. 

Within health facilities, observations of time spent by 
clients at different service points and in total, prescrip- 
tion content analysis, and client exit interviews were car- 
ried out using observation checklists, interview guides, 
and semi-structured questionnaires. The client exit in- 
terviews had a mix of closed and open ended items to 
explore client experience in the clinic related to service 
supply and responsiveness, previous experiences, opin- 
ions about the NHIS, and suggestions for making the 
NHIS more responsive. 


Sampling 

Sampling was purposive. Participants for the national level 
key informant in-depth interviews were selected from the 
list of designers of the G-DRG payment method [28]. For 
regional and district primary data collection, we stratified 
the country into three zones of relatively similar socio- 
economic characteristics, namely Northern (Upper East, 
Upper West, and Northern regions); Central (Brong Ahafo 
and Ashanti regions), and Southern (Volta, Eastern, Greater 
Accra, Central, and Western regions). Within the Central 
zone the Brong Ahafo region was purposively selected be- 
cause the on-going pilot of capitation in the Ashanti region 
would make it difficult to evaluate incentives inherent in 
the nationwide payment systems as against capitation pilot 
effects. Within the Southern ecological zone, the Greater 
Accra region was purposively selected because of its peculi- 
arity of being 90% urban with a large and active private 
self-financing provider community and the lowest average 
poverty levels in the country. Within the three Northern re- 
gions, which have the highest percentage of rural popula- 
tions and poverty levels in the country, the Upper West 
Region was randomly selected by balloting since there was 
no clear rationale to justify purposive selection. 

Within each of the three regions, a list of the most 
recent local government demarcation and classification 
of districts with districts stratified into rural, municipal, 
and metropolitan was obtained. One district in each cat- 
egory was selected per region by balloting. Greater Accra 
was the only region with metropolitan districts and a 
sub-metropolis in Accra was selected by balloting. The 
NHIA office covering each selected district was included 
in the sample. 

In the selected districts, a list of government, CHAG, 
and private self-financing facilities was obtained from the 
Ghana Health Service and one NHIA accredited district 
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Table 1 Geographic location of primary data collection (level, facility type, and ownership), method of data collection, 


and numbers of respondents 


Level (National, Regional, ‘dell 
District/Municipal/ p 
Sub-metro) _ _ ae a. —aae—a 
a Facility name Facility type Ownership Key informant Client exit Client . 
in-depth interview waiting time 
interview assessment 
National Not applicable Not applicable Not applicable 14 0 0 
Ashiedu Keteke Ussher polyclinic Polyclinic Public (GHS) Zz 35 35 
sub-metro 
PML Children’s hospital Hospital Public (GHS) 0) 33 34 
Cathedral clinic Clinic Private (PSF) 0 35 25 
Ashiedu Ketete district Purchaser Public (NHIA) 1 0 0 
scheme 
La Dadekotopon La General hospital Hospital Public (GHS) 2 35 37 
sub-metro 
Madina La Nkwantang Pentecost hospital Hospital Private (CHAG) 2 35 34 
municipality 
North Legon hospital Hospital Private (PSF) 2 31 ZZ 
Madina Polyclinic Polyclinic Public (GHS) Z 35 35 
(New Road) 
Ga municipal scheme Purchaser Public (NHIA) 1 0 0 
Ada East District Ada HC Health centre Public (GHS) ) 35. 35 
Pute CHPS CHPS Public (GHS) 2 5 4 
TOTAL Greater Accra 30 279 261 
Region 
Dorma municipality Amasu HC Health centre Public (GHS) 2 10 11 
Twumkrom CHPS CHPS Public (GHS) 2 0 0 
Dorma Presby Hospital Hospital Private (CHAG) 2 30 Z3 
Saviour clinic Clinic Private (PSF) ] 13 13 
Dormaa district scheme Purchaser Public (NHIA) 1 
Dorma West district Nkrankwantakrom Health centre/DH Public (GHS) 2 29 39 
Kojo Kumi Krom Health centre Private (CHAG) 29 
Yaa Krom HC Health centre Private (CHAG) 2 16 2 
Kwakuanya Ebenezer Clinic Private (CHAG) 1 0 0 
Methodist 
Kyremesu Presby HC Health centre Private (CHAG) 2 3 18 
Kwabenadwomo Health centre Public (GHS) 2 13 0 
Bedi Brong Ahafo 17 143 106 
egion 
Sisala East district Tumu hospital Hospital Public (GHS) 2 23 37 
Wellembele HC Health centre Public (GHS) "4 16 19 
Nnamdouonu CHPS CHPS Public (GHS) ? 7 8 
Mama Mary Clinic Private (PSF) 2 18 22 
Sissala East district scheme Purchaser Public (NHIA) 1 0 0 
Wa municipality Kambali HC Health centre Public (GHS) Z 30 SF 
Tampalipani CHPS CHPS Public (GHS) 2 4 4 
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Table 1 Geographic location of primary data collection (level, facility type, and ownership), method of data collection, 


and numbers of respondents (Continued) 


islamic hospital Hospital 
Wa Municipal scheme Purchaser 
TOTAL Upper West 
Region 
GRAND TOTAL 


Private (PSF) 2 35 26 32 

Public (NHIA) 1 0 0 0 
16 145 153 159 
63 567 520 528 


GHS, Ghana Health Service; PSF, Private Self-Financing; CHAG, Christian Health Association of Ghana; CHPSC, Community Health Planning and Services Compound; 


NHIA, National Health Insurance Authority. 


hospital, health centre, and CHPS compound in the public 
sector were selected by balloting. Where the district had 
CHAG and private self-financing facilities, one CHAG 
and one private facility were selected by balloting if there 
was more than one; if there was only one it was selected. 
During data collection, some selected facilities had to be 
substituted with the nearest facility of the same category 
because the information in the national level facility list- 
ings did not always reflect what was happening at the 
frontline and the selected facility was no longer functional. 

Data collectors visited each clinic starting at the morn- 
ing shift hours of 8.00 am. All clients entering the clinic — 
regardless of insurance status — were tracked for time 
spent at the different service points until a maximum of 
35 clients was reached. Some of the smaller clinics had 
low client loads and it was not possible to get 35 clients in 
one day but the time frame and budget of the study did 
not allow repeat visits. Prescriptions issued to these cli- 
ents were copied for analysis, and an exit interview 
administered. Ethical clearance was obtained from the 
Ghana Health Service Research and Development div- 
ision; all tracking and interviewing was done with in- 
formed consent. 


Data analysis 

The study was carried out in response to a request by 
the Ghana NHIA for an evaluation of its DRG payment 
methods. The constraints of the time frame of the re- 
quest meant that the data collection and analysis was 
done using overlapping processes. The team had mixed 
qualitative and quantitative data collection and analysis 
skills among the members. The same two members of 
the research team did all the national level in-depth in- 
terviews and, together with a third team member, they 
performed the qualitative analysis. For the district level 
primary data collection, the research team split into 
three groups to collect data with support from research 
assistants. 

Apart from notes during the interview, in-depth inter- 
views were recorded and transcribed. Analysis was man- 
ual to identify themes, commonalities, and contrasts. 
The open-ended questions in the exit interviews were 
typed into Excel, coded, and classified by themes and 
sorted for analysis. Primary quantitative data analysis 


was done in Excel and Stata. Routine health manage- 
ment information system data was analysed in Excel. We 
used frequencies, cross tables, and trend lines for the 
quantitative data analysis. Team members were assigned 
responsibility for analysing particular data sets depend- 
ing on their expertise. 

To help generate a more holistic theory as to the rela- 
tionships between provider payment methods, service 
supply incentives and behaviour, and what mechanisms 
explain these observed effects, we used causal loop and 
causes tree diagrams [29] — both systems thinking tools. 


Validity, quality assurance, and limitations 

Several methods were used to ensure validity. Firstly, we 
have presented our methods in detail to enable readers to 
judge the quality of the data. Secondly, the whole team 
discussed analysis and conclusions from each data set, and 
findings from different data sets related to the same ques- 
tion were compared as part of triangulation and minimiz- 
ing individual team member biases. This also allowed a 
more reflexive approach to data analysis. Thirdly, we paid 
attention to extreme as well as middle cases in our ana- 
lysis and did not focus on frequently repeated responses 
only. Fourthly, before finalizing our report, we organized 
two different half-day validation meetings with representa- 
tives of our respondents to present our initial analysis and 
conclusions and obtain their feedback. This was part of 
the iterative process of data collection and data analysis. 
We also made our draft report available to respondents 
who were willing to read it, to check if it was valid from 
the perspective of their experiences that we were trying to 
describe and analyse. 


Findings and discussion 

We used trends in utilization for insured and uninsured 
to assess changes in supply related to the number of 
clients seen by providers. The data is summarized graphic- 
ally in Figures 4, 5, and 6. Both the provider and purchaser 
data sets tell the same story of increased numbers of visits 
per active insured member for inpatient and outpatient 
services. There does not appear to have been a similar 
change over time in the number of visits to formal pro- 
viders for the uninsured. Data was not available to enable 
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Figure 4 Outpatient (OP) and inpatient (IP) claims per active 
member per annum (NHIA routine management information 
system data). 


an assessment and comparison of trends in visits to non- 
formal providers. 

The insurance status of the clients participating in our 
exit interviews showed the same pattern of a dominance 
of utilization of formal services by the insured. Of the 
total of 567 clients in the exit interviews, 432 (76%) had 
a valid insurance ID card. 

It is, however, difficult to come to any firm conclusion 
from the data available that this utilization reflects in- 
centives for providers to preferentially see insured clients 
or is due to service provider (supplier)-induced demand. 
There are several possible reasons related to demand as 
well as supply that could explain the data. These include 
the increases in utilization among the insured reflecting 
increased client demand induced by the removal of the 
financial barrier and frivolous use by insured clients also 
related to the removal of the financial barrier. It could, 
however, also reflect some supplier-induced demand. Sev- 
eral respondents in our national level qualitative inter- 
views mentioned that, for any given illness episode, the 
G-DRG design allows a provider to bill for three visits for 
outpatient care — the initial visit and two follow-up visits. 
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It could be to the financial advantage of the provider to 
bill routinely for all three visits regardless of whether the 
client needed or even made them. 

Making it even more difficult to conclude on supplier- 
induced demand as a major reason for the increased 
numbers of encounters per insured client is that our key 
informant interviews with frontline providers suggested 
that the G-DRG is leading to some shifting of cases in 
the form of referral. The quote below illustrates this as 
well as the impression we got that there was a disincen- 
tive to see certain kinds of insured clients because pro- 
viders felt the reimbursements were inadequate for the 
inputs needed to manage the case. 


“..you can imagine somebody bringing an ulcer... you 
know that (dressing) a big sore daily... the cost will go 
up so you will lose... so we were losing, so that was 
why most of us were not dressing this thing, we refer 
them to the hospital... yes, even the suturing too was a 
problem; the money was just a token.” Rural Health 
Centre nurse 


The actual as well as perceived inadequacy of the re- 
imbursement rates were compounded by the delays in 
reimbursements. To illustrate with the words of a hos- 
pital medical superintendent: 


“... The payment system has really broken down to a 
certain extent. They are not consistent with the 
payments and it is disturbing our work. It makes us 
financially not sound... Promptness is the bigger 
problem rather than the rate... If they would pay us 
promptly I would be so happy.” 


These observations lead to other findings on the di- 
mension of supply related to input use per client. The 
indicators explored to help understand this dimension 
of provider supply decisions were volume of tests and 
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Figure 6 Provider routine management information system 
data trends in inpatient (IP) visits insured and uninsured. 


procedures, medicines prescribing, and client time spent 
in facilities. 


Volume of diagnostic tests and procedures 

Some of the responses obtained from providers and cli- 
ents suggest that the bundled payments of the G-DRG 
for services were a disincentive to carry out extensive 
diagnostic investigations whether they were needed or 
not, for example: 


“..the grouped billing... is a disincentive to carry 
extensive investigations” Pharmacist, Urban Polyclinic 


An NHIS subscriber described how he presented for 
services without showing his insurance card. After his 
history and examination, he was asked to do several la- 
boratory tests to help confirm a diagnosis. At this point, 
he mentioned he had an insurance card and asked if that 
could cover his treatment including the laboratory tests. 
There was a subtle change in the facial expression of the 
staff and he was asked why he had not provided this 
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information earlier. He was then asked to please give his 
folder back for review and wait. After a while, the folder 
was brought back to him with the laboratory tests can- 
celled, and the information that they were not needed. 
He could just go and collect his medicines. 


Medicine prescribing 

The literature review of the incentives associated with 
different payment methods suggests that over or at least 
adequate provision would be an incentive for the supply 
of medicines under the NHIS, given that an itemized fee 
for medicine schedule is the payment method. Our data 
sometimes suggested, but was not always convincingly 
in support of, a situation of adequate or over rather than 
under provision of medicines. The average number of 
medicines per outpatient prescription in our sample was 
four for all prescriptions (n = 527)*, three for the non- 
insured (n = 98), and four for the insured (n = 429). The 
most recent national data on prescribing indicators avail- 
able for comparison was the pharmaceutical situation 
assessment carried out in May/June 2008 [30]. It unfortu- 
nately did not compare data between insured and unin- 
sured. The average number of medicines per prescription 
in this survey was four. 

Other prescribing indicators from our survey are sum- 
marized in Figure 7 and compared with data from the 
2008 national survey. There was no data on whether 
medicines were on the NHIML in the 2008 survey. It 
would be expected that facilities would supply nearly 
100% of the medicines prescribed from their dispensaries 
since, in theory, the more medicines supplied, the more 
income the provider earns. In our survey, however, only 
78.7% of medicines prescribed were dispensed in the fa- 
cility. A higher percentage was dispensed in the facility 
to the insured (81.6%) as compared to the uninsured 
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(64.4%). However, as Figure 7 shows, compared to the 
2008 national survey data where 94.2% of all medicines 
prescribed were dispensed in the facility, the percentage 
of medicines prescribed that were dispensed in the facil- 
ity, whether to insured or uninsured, is low. It raises 
questions as to whether something other than insurance 
status and provider payment method is modifying pre- 
scribing and dispensing behaviour. 

The NHIS policy requires that medicines are pre- 
scribed by generic name and are on the NHIML, other- 
wise they do not qualify to be supplied “free” under the 
NHIS. As Figure 7 shows, there was higher prescribing 
of medicines by generic name and from the NHIML for 
insured as compared to uninsured clients. It is reason- 
able to suspect that this is an influence of the provider 
payment method and its associated rules. However, in 
spite of these rules, not all medicines for insured clients 
were prescribed by generic name or listed on the NHIML. 
It would appear that something else is also influencing be- 
haviour. The influence is probably bigger for the unin- 
sured who do not have the effect produced by the rules of 
the NHI provider payment method. Our data does not 
allow a complete explanation of what is causing these pre- 
scribing and dispensing behaviour gaps, which are wider 
for the uninsured than the insured. We can only make 
some guesses based on our observations. One of them is 
that providers repeatedly complained that the reimburse- 
ment rates for some medicines on the NHIML are too 
low. Perhaps, the actuality as well as the perception of too 
low tariffs could negate, in part, any incentives to pre- 
scribe and dispense such medicines. Secondly, weaknesses 
in the implementation of the rational use of medicines 
policy in Ghana could also explain the uninsured data. 
The rational use of medicines policy requires that all med- 
icines are prescribed by generic name and are listed on 
the Essential Medicines List. The Essential Medicines List 
and the NHIML overlap, but not completely. Finally, the 
condition of some patients may just have required medi- 
cines to be prescribed outside the NHIML and the Essen- 
tial Medicines List, and which were not available as 
generic. It is doubtful, however, if such cases at the out- 
patient primary care level should account for as many as 
10 to 25% of prescriptions. 

The exit interview data confirmed the failure to supply 
all medicines prescribed in the facility dispensary and 
also showed insured client discontent with this failure. 
Of the 41 clients in the exit interviews (7% of the total 
sample) who felt the NHIS was bad, a little over half 
(22) gave a reason related to the failure to supply all 
medicines prescribed free as part of their NHIS benefits. 
Examples typical of these responses include: 


“The aspect where the scheme does not cover all the 
drugs is worrying to us especially we the poor...” 
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“They do not give all the drugs” 


“Buying drugs outside the hospital while you still have 
a valid insurance” 


« dislike the NHIS because initially it was supposed 
to be free but now I’m made to buy drugs anytime” 


Some out of pocket payments by insured clients is not 
a new finding; as many as 94% of respondents in the 
Ghana Demographic and Health Survey [13] reported 
sometimes making out of pocket payments for medi- 
cines, services, or both. The SHINE project also docu- 
mented insured client out of pocket payments. They 
were, however, significantly lower than uninsured client 
payments [31,32]. Some of these out of pocket payments 
are for services and medicines not covered by the NHIS. 
Others are related to managerial inefficiencies, e.g., 
stock outs, and others to reluctance to stock and supply 
NHI clients with items that are seen as potentially caus- 
ing financial loss to the provider because of unattractive 
NHL tariffs. 


Time spent with patients 

It was not clear that time spent in the clinics by clients 
was related to provider payment incentives. Client loads 
and staffing constraints rather appeared to be the influ- 
ences over it. Figure 8 summarizes total time spent by 
facility. The longest waiting times were in the crowded 
mission (CHAG) and public (Ghana Health Service) hos- 
pitals, polyclinics and health centres. The private hospitals 
and clinics and the CHPS compounds were we recorded 
the lowest times spent by patients in getting care were also 
the facilities in which we observed lower client numbers. 


Pulling it all together 

We have qualitatively explored some answers to questions 
related to the “what” and the “why” of service supply 
behaviour and incentives related to provider payment 
methods of the Ghana NHIS. To answer the “what” ques- 
tion, we have conceptualized and described service supply 
behaviour in two dimensions of numbers of insured cli- 
ents and inputs into management of individual clients 
seen. T’o answer the “why” question, we have drawn upon 
systems thinking and realist evaluation theories and exam- 
ined the wider national context, the health system, and 
their influence. We now pull these pieces together to gen- 
erate potential explanatory theory employing the systems 
thinking tool of causal loop and causes tree diagrams 
qualitatively. Figure 9, a causal loop diagram, is our con- 
cluding theoretical model. Since the diagram is qualita- 
tively constructed, it provides no indication of magnitude 
of effect. 
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Figure 8 Total time spent by facility. 
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As for any causal loop diagram the theorized direction 
of influence of one factor on the other is show by the 
direction of the arrow. The labels at the tips of the arrows 
indicate whether the influence is to cause a decrease or in- 
crease in the level of the influenced factor. An “S” sign 
against the arrow head means that, as the causal variable 
increases or decreases, the influenced variable changes in 
the same direction. An “O” sign means the change is in 
the opposite direction. Where there is neither an “O” or 
an “S” sign, the relationship is something other than a 
straight forward increase or decrease. 

Thus, for example in Figure 9, we theorize that the 
amount of internally generated funds (IGF) available for 
immediate use through NHI payments influences incen- 
tives to supply service to insured clients. IGF is a term 
used in Ghana to describe funds that are generated and 
retained for use at the facility level as compared to ‘ex- 
ternal’ funds such as Central Government allocations. 
IGE comes from NHI payments, out of pocket fees, and, 
in a few instances, reimbursements from corporations 
and agencies on behalf of their workers. For private 


sector facilities it is their entire source of income. For 
public sector facilities it forms 80% or more of their in- 
come for recurrent expenditure [33]. The more IGF is, 
the more incentivized providers are to supply services to 
insured clients in both dimensions. Out of pocket pay- 
ments have an immediate effect on IGF availability un- 
like NHIS payments whose effects are modified by the 
time lag between claims submission and claims payment. 
Additionally, IGF availability from insured clients is af- 
fected by the perceived and actual complexity of claims 
processing procedures of provider and purchaser. Com- 
plex procedures tend to take more time to fulfil and can 
increase the time lag to final payment. They also require 
more skilled staff numbers and time and may be more 
likely to lead to mistakes in claims submission by pro- 
viders as well as auditing by the purchaser that lead to 
denial of claims. 

Causal loop diagrams are difficult to follow for those 
who have not been involved in the details of construct- 
ing them. To make the causal loop diagram easier to fol- 
low, we have unpacked its core into a series of cause 
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L Figure 9 Causal loop diagram. 


tree diagrams in Figures 1OA—E. Figure 10A shows factors 
that we theorize from our observations to have a direct in- 
fluence on supply incentives. These are IGE available for 
immediate use, level of general budget support provided 
by government for service delivery, personal provider in- 
come per client encounter, and workload. Factors that dir- 
ectly influence each of these factors are unpacked in cause 
tree diagrams 10B—E. 

There does not appear to be a linear relationship be- 
tween provider payment method and incentives to see 
more active NHIS card holders or provide more inputs 
per client. The relationships are indirect and modified 
by contextual factors. Some of the different factors are 
controlled by different agents or actors, with some con- 
trolled by multiple agents. A reinforcing loop in the causal 
loop diagram is the link between institutional functioning 
and IGF available for immediate use. We theorized that 
better-managed facilities might be better able to find ways 
of coping with the resource constraints of the health sys- 
tem. Improved resource availability was likely to reinforce 
better management. 

The observed status of many of the variables would 
produce a tendency to prefer out of pocket paying cli- 
ents to clients who are paid for by insurance and to con- 
tract some but not all service inputs to insured clients. 
Our data does not allow us to answer the question of 
whether the current levels of inputs per client are ad- 
equate. More services do not necessarily translate into 
high quality and responsiveness. However, it is doubtful 
if a tendency to incentivise contraction of service inputs 
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in a system of resource scarcity will ensure high quality 
and responsiveness. 


Conclusions 

In our study setting, service supply behaviour and the 
incentives driving it cannot necessarily be predicted in 
abstract from the theory about the anticipated response 
to a given payment method. The wider national context 
as well as characteristics of the health system into which 
the method is introduced shape and modify supply be- 
haviour and incentives. This is not surprising given that 
the payment method reforms have been introduced into a 
complex adaptive system. The individual agents (whether 
institutions, persons or groups) in such systems are inter- 
connected and have the freedom to act in ways that are 
not always predictable. Whether ignored or acknowl- 
edged, complexity remains and affects outcomes in such 
systems. To be able to cope, it is better to recognize, 
understand and try to work with complexity rather than 
engage in the futile effort of trying to “reduce” it with 
linear approaches [34,35]. 

Provider payment reform in low- and middle-income 
countries should pay at least as much, if not a little 
more, attention to context of the reform and the poten- 
tial interactions between the reform and context and 
the resulting intended and unintended effects as to the 
method itself in the design and implementation of 
reform. 

Finally, Gauri [36] has observed that “data limitations, 
selection effects and numerous confounding variables” 
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| Figure 10 (A) Factors influencing service supply incentives. (B) Factors influencing IGF available for immediate use. (C) Factors influencing 
| personal income per client encounter. (D) Factors influencing workload. (E) Factors influencing level of direct government budget support 

| to providers. 
Nalliei 


make study difficult and limits the availability of empir- | Endnote 

ical research on provider payment mechanism effects ‘Prescriptions transcribed with missing data dropped 
on providers in low- and middle-income countries. We from the analysis accounts for the difference between 
agree from our experience and make a plea for con- the total number of prescriptions in the sample (528) 
tinued work on methodological approaches in such and the number of prescriptions from which analysis is 
settings. presented (527). 


ADVANCING THE APPLICATION OF SYSTEMS THINKING IN HEALTH 131 


132 


Agyepong et al. Health Research Policy and Systems 2014, 12:35 
http://www. health-policy-systems.com/content/1 2/1/35 


Abbreviations Ta | 
CAS: Complex Adaptive Systems; CHAG: Christian Health Association of 
Ghana; CHPS: Community Health Planning and Services; G-DRG: Ghana 
Diagnostic Related Groupings; IGF: Internally Generated Funds; NHI: National 
Health Insurance; NHIA: National Health Insurance Authority; NHIML: 
National Health Insurance Medicines List; NHIS: National Health Insurance; 
SSNIT: Social Security and National Insurance Trust; UHC: Universal 


Health Coverage. 


Competing interests 
The authors declare that they have no competing interests. 


Authors’ contribution 

All authors contributed to study design, data collection, and analysis. [AA 
conceptualized and wrote the paper. All authors read and commented on 
the draft. All authors read and approved the final manuscript. 


Acknowledgements 

The study from which the material in this paper is derived received financial 
support from the National Health Insurance Authority through the World 
Bank Health Insurance Project. The authors are solely responsible for all 
opinions and conclusions presented in this paper. 

This paper is part of the Thematic Series entitled: “Advancing the application 
of systems thinking in health”. The Series was coordinated by the Alliance for 
Health Policy and Systems research, World Health Organization. The 
publication of the Series and the associated capacity building and 
dissemination activities were carried out with the aid of a grant from the 
International Development Research Centre, Ottawa, Canada. 


Author details 

"Department of Health Policy Planning and Management (HPPM), University 
of Ghana School of Public Health, P.O. Box LG 13, Legon, Accra, Ghana. 
*National Health Insurance Authority, No. 36, 6th Avenue, Opposite AU Suite, 
Ridge Residential Area, Accra. Private Mail Bag, Ministries, Accra, Ghana. 
*Private Consultant, P.O. Box DS 331, Dansoman Estates, Accra, Ghana. 
“Ghana Health Service Greater Accra Regional Health Directorate, P.O. Box 
184, Accra, Ghana. °Korle-Bu Teaching Hospital, P.O. Box 77, Accra, Ghana. 
°Ghana Health Service, Policy, Planning, Monitoring and Evaluation Division 
(PPMED), Private Mail Bag, Ministries, Accra, Ghana. Noguchi Memorial 
Institute for Medical Research, University of Ghana, P.O. LG 581, Legon, Accra, 
Ghana. 


Received: 6 January 2014 Accepted: 23 June 2014 
Published: 5 August 2014 


References 

1. WHO (2005). Sustainable health financing, universal coverage and social 
health insurance, World Health Assembly Resolution 58.33 (2005). http://apps. 
who.int/iris/bitstream/10665/20383/1 /WHAS8_33-en.pdf?ua=1. 

2. _ UN (2012). United Nations General Assembly, Sixty seventh session, Agenda item 
123. Global Health and Foreign policy A/67/1.36. http://nedalliance.org/sites/ 
defau't/files/resource_files/Global Health and Foreign Policy resolution 201. 

3. World Health Organization: The World Health Report 2010. Health Systems 
Financing. The Path to Universal Coverage. Geneva: WHO: 2010. 

4. Kutzin J: A descriptive framework for country level analysis of health care 

financing arrangements. Health Policy 2001, 56:171~204, 

Jegers M, Kesteloot K, De Grave D, Gilles W: A typology for provider 

payment systems in health care. Health Policy 2002, 60:255-273, 

6. Paina L, Peters DH: Understanding pathways for scaling up health 
services through the lens of complex adaptive systems. Health Policy Plan 
2012, 27(5):365-373. 

7. de Savigny D, Adarns T: Systems Thinking for Health Systems Strengthening. 
Geneva: Alliance of Health Policy and Systems Research, WHO; 2009, 

8. Senge P: he Fifth Discipline. The Art and Practice of the Learning 
Organization. New York: Crown Publishing Group; 1990, 

9. Index Mundi; 2013 (http//www.indexmundi.com/facts/ghana/gni-per-capital 

10. Ghana Inflation Rates 1992-2013: Ghana Inflation Rates 1992-2013 
[http://datablog.peacefmon! ine.com/pages/blog/9/] Accessed on 
6 April 2014. 

11. Index Mundi; 2013. [http//www.indexmundi.com/ghana/inflation_rate_ 
(consumer_prices).html] Accessed on 6 April 2014. 


On 


HEALTH RESEARCH POLICY AND SYSTEMS 


Boi 


20. 


30, 


31. 


EWR 


34, 


Page 16 of 17 


SSNIT Annual Report 2011. Accra: SSNIT Pension House [http//www.ssnit.com] 
Ghana Statistical Service (GSS), Ghana Health Service (GHS), and ICF 
Macro: Ghana Demographic and Health Survey, Accra: GSS, GHS and ICF 
Macro; 2008, 

Bhutta ZA, Chopra M, Axelson H, Berrnan P, Boerma T, Bryce J, Bustreo 
F, Cavagnero E, Cometto G, Daelmans B, de Francisco A, Fogstad H, 
Gupta N, Laski L, Lawn J, Maligi B, Mason E, Pitt C, Requejo J, Starrs A, 
Victora CG, Wardlaw T: Countdown to 2015 decade report (2000-10): 
taking stock of maternal, newborn, and child survival. Lancet 2010, 
375:2032-2044., 

WHO: Working Together for Health. The World Health Report 2006. 
[http://whqlibdoc.who.int/publications/2006/924 1563176_eng.pdf] Accessed 
on 25 October 2013 

Ghana Ministry of Health: Holistic Assessment of the Health Sector 
Program of Work 2012, 2013. [http://www.moh-ghana.org/UploadFiles/ 
Publications/2012%20Holistic%20Assessment%20Report% 
200foe 1 30715062103,pdf] Accessed on 25 October 2013. 

Ghana Ministry of Health: National Health Insurance Policy Framework for 
Ghana. Accra: MOH; 2002. 

Ghana Ministry of Health: National Health Insurance Policy Framework for 
Ghana. Revised Version. Accra: MOH; 2004. 

Agyepong IA, Adjei S: Public social policy development and 
implementation: a case study of the Ghana National Health Insurance 
scheme. Health Policy Plan 2008, 23(2):150-160. 

Mcintyre D, Garshong B, Mtei G, Meheus F, Thiede M, Akazili J, Ally M, 
Aikins M, Mulligan JA, Goudge J: Beyond fragmentation and towards 
universal coverage: insights from Ghana, South Africa and the 
United Republic of Tanzania. Bull World Health Organ 2008, 

86(1 1):871--876. 

Ansah EK, Narh-Bana S, Asiamah S, Dzordzordzi V, Biantey K, Dickson K, 
Gyapong JO, Koram KA, Greenwood BM, Mills A, Whitty CJ: Effect of 
removing direct payment for health care on utilisation and health 
outcomes in Ghanaian children: a randomised controlled trial. PLoS Med 
2009, 6(1):e 1000007. Erratum in: PLoS Med 6(2):e 1000033. 

Witter S, Garshong B: Something old or something new? Social health 
insurance in Ghana. BMC Int Health Hum Right 2009, 9:20. 

Mathauer |, Wittenbecher F: DRG-Based Payment Systems in Low and Middle 
Income Countries: Implementation Experiences and Challenges. Discussion 
paper number 1 ~ 2012. Department “Health Systems Financing” (HSF) Cluster 
‘Health Systems and Services” (HSS). Geneva: WHO; 2012. 

Hesse A: National Health insurance Scheme, Tariff Development Committee 
Main Report. November 2007. Unpublished consultant report. Accra: Executive 
Health Care and Consult Ltd; 2007. 

Sodzi-Tettey S, Aikins M, Awoonor-Williams K, Agyepong IA: 
Challenges in provider payment under the Ghana NHIS: a case 
study of claims management in two districts. Ghana Med / 2012, 
46(4):189-199, 

Kipp R, Baveja L: Developing Nations Can Gain From Better Bundled Payment 
information, Milliman White Paper, 2013 [http://publications.milliman.com/ 
publications/health-published/pdfs/ghana-drg.pdfi 

Pawson R, Tiley N: Realistic Evaluation. London: Sage; 1997. 

Hesse A: National Health insurance Scheme. Tariff Development Committee 
Main Report. Unpublished consultant report. NHIA: Accra; 2007. 

Chapter 1. Systems Behavior and Causal Loop Diagrams: Chapter 1. Systems 
Behavior and Causal Loop Diagrams. {http://www.public.asu.edu/~kirkwood/ 
sysdyn/SDintro/ch-1 pdf] Accessed 18 December 2013. 

Arhinful DK: Ministry of Health Ghana WHO Pharmaceutical Situation 
Assessment Level Il Health Facility Survey in Ghana; 2009 [http://Awww.moh- 
ghana.org/Uploadtiles/Publications/Ghana_baseline_WHO_level_Il_Health_ 
Facility_Survey140204075637.pdf} 

Jehu-Appiah C, Aryeetey G, Spaan £, de Hoop Tf, Agyepong |, Baltussen R: 
Equity aspects of the National Health Insurance Scheme in Ghana: who 
is enrolling, who is not and why? Soc Sci Med 2011, 72(2):157-165. 
Jehu-Appiah C, Aryeetey G, Spaan E, Agyepong |, Baltussen R: Household 
perceptions and their implications for enrolment in the National Health 
Insurance Scheme in Ghana. Health Pol Plan 201 2, 27222233. 

Agyepong IA, Nagai RA: “We charge them; otherwise we cannot run 
the hospital” front line workers, clients and health financing policy 
implementation gaps in Ghana. Health Policy 2011, 99:226-233. 

Plsek PE, Greenhalgh T: The challenge of complexity in health care. BM/ 
2001, 323:625-628. 


Agyepong et al. Health Research Policy and Systems 2014, 12:35 
httpy/www.health-policy-systems.com/content/12/1/35 Page 17 of 17 


35. Pisek PE, Wilson T: Complexity, leadership, and management in 
healthcare organizations. BMJ 2001, 323:746-749. 

36. Gauri V (2001). Are Incentives Everything? Payment Mechanisms for Health 
Care Providers in Developing Countries. NW, Washington, DC: World Bank 
Policy Research Working Paper Series #2624. |http:/Awww-wds.worldbank. 
org/servieYWDSContentServer/WDSP/iB/2004/01/30/0002655 13 
200401 30094539/Rendered/PDF/wps2624.pdf] F 


doi:10.1186/1478-4505-12-35 
Cite this article as: Agyepong et al: Advancing the application of 
systems thinking in health: provider payment and service supply 


behaviour and incentives in the Ghana National Health Insurance 


Scheme — a systems approach. Health Research Policy and Systems 
2014 12:35. 


Submit your next manuscript to BioMed Central 
and take full advantage of: 


° Convenient online submission 

° Thorough peer review 

° No space constraints or color figure charges 
¢ immediate publication on acceptance 


° inclusion in PubMed, CAS, Scopus and Google Scholar 


° Research which is freely available for redistribution 


Submit your manuscript at 4 , BioMed Central 


www.biomedcentral.com/submit 


ADVANCING THE APPLICATION OF SYSTEMS THINKING IN HEALTH 4.33 


HEALTH RESEARCH POLICY 
AND SYSTEMS 


Zhang et al. Health Research Policy and Systems 2014, 12:44 
http://www. health-policy-systems.com/content/1 2/1 /44 


of 
<— 


Uy 


Y, 


RESEARCH Open Access 


Advancing the application of systems thinking in 
health: managing rural China health system 
development in complex and dynamic contexts 


Xiulan Zhang", Gerald Bloom, Xiaoxin Xu’, Lin Chen!, Xiaoyun Liang! and Sara J Wolcott’ 


Abstract 


Background: This paper explores the evolution of schemes for rural finance in China as a case study of the long 
and complex process of health system development. It argues that the evolution of these schemes has been the 
outcome of the response of a large number of agents to a rapidly changing context and of efforts by the 
government to influence this adaptation process and achieve public health goals. 


Methods: The study draws on several sources of data including a review of official policy documents and academic 
papers and in-depth interviews with key policy actors at national level and at a sample of localities. 


Results: The study identifies three major transition points associated with changes in broad development strategy 
and demonstrates how the adaptation of large numbers of actors to these contextual changes had a major impact 
on the performance of the health system. Further, it documents how the Ministry of Health viewed its role as both 
an advocate for the interests of health facilities and health workers and as the agency responsible for ensuring that 
government health system objectives were met. It is argued that a major reason for the resilience of the health 
system and its ability to adapt to rapid economic and institutional change was the ability of the Ministry to provide 
overall strategy leadership. Additionally, it postulates that a number of interest groups have emerged, which now 
also seek to influence the pathway of health system development. 


Conclusions: This history illustrates the complex and political nature of the management of health system 
development and reform. The paper concludes that governments will need to increase their Capacity to analyze the 
health sector as a complex systern and to manage change processes. 


Keywords: Health system, Institutional development, Rural health finance, System resilience 


Background but provided timely and low cost treatment to rural resi- 
China created a system of rural health care finance dur- _ dents [1]. They were supervised by medical doctors and 
ing the period of the socialist planned economy (1949 to they could refer patients to county hospitals. This rural 
1978), when the level of economic development was very health system contributed to significant improvements 
low, and a large proportion of the population lived in in access to basic health services and in health outcomes 
poverty. The Cooperative Medical System (CMS) pro- [2]. This rural health finance system collapsed during 
vided basic health benefits to most rural farmers. By China's transition to a market economy, which began in 
1976, more than 90% of rural villages had a CMS anda __‘1978. During the 1980s and 1990s, efforts were made to 
network of preventive and curative health facilities at rebuild it, with little success. However, starting in 2002, 
county, township, and village levels. Most services were the government began to implement so-called New Co- 
provided by “barefoot doctors”, who had limited training operative Medical Schemes (NCMS) and, by 2012, 805 
million rural residents, or 98% of the rural population, 
* Correspondence: zhang99@bnu.edu.cn : ere oo Oe es pas SI. 
‘School of Social Development and Public Policy (SSDPP), Beijing Normal This paper explores the evolution of CMS and NCMS 


University, 19 Xinjiekouwai Street, Beijing 100875, China since the late 1970s as a case study of the long and 
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complex process of health system development. It argues 
that its trajectory has been the result of the response of 
large numbers of agents to a rapidly changing context 
and of efforts by the government to influence this adap- 
tation process. It draws on the concept of resilience, 
which Walk et al. [4] define as “the capacity of a system 
to absorb disturbance and reorganize while undergoing 
change so as to still retain essentially the same function, 
structure, identity, and feedbacks”. It concludes that the 
way that the government manages the adaptation of the 
health system to rapid change strongly influences the de- 
gree to which it meets socially agreed objectives of pro- 
viding access to safe and effective services for all [5]. 


Applying system thinking to health care systems: 

a literature review 

Understanding health care systems as complex adaptive 
systems 

There is a growing interest in applying concepts of com- 
plex adaptive systems (CAS) [6-14] to the analysis of 
health systems. A CAS has many components, often 
called agents, which interact in apparently random ways 
[15]. Through these interactions, patterns emerge and the 
system is continually self-organizing through processes of 
emergence and feedback. Agents in the system are ignor- 
ant of the behavior of the system as a whole, responding 
only to local information [16-19]. Policy-makers, who 
want to implement a new policy and overcome resistance 
[13], need to pay attention to the context, the behavior 
and networks of agents, and likely feedback loops [20]. 

A number of case studies in the advanced market econ- 
omies have applied CAS concepts to different aspects of 
health system performance, such as disease control [21], 
nursing homes [22], palliative care [23], family practices 
[24], and primary care [25-28], in designing evaluation re- 
search [29-33], in interpreting research findings [34,35], 
and in other health system areas [36-39]. These studies 
have increased our understanding of the role and behavior 
of agents involved in health systems. 

There are a growing number of studies of health sys- 
tems in low- and middle-income countries. Many of these 
countries have weak institutional arrangements in com- 
parison to the advanced market economies and this influ- 
ences how policy is translated into changes in system 
behavior [40]. Xiao et al. [41] explore this with regard to 
China’s implementation of an essential drugs policy. They 
demonstrate that the interaction of responses by different 
actors has led to divergent and unpredicted outcomes. 
They conclude that the management of policy change in 
rural China needs to take into account the emergence of 
adaptive and self-organized behavior and that many 
changes are nonlinear. 

Paina and Peters [42] offer a macro perspective on 
strategies for taking health system changes to scale. They 
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examine the impact of system history, context, and polit- 
ical and institutional structures on the complex path- 
ways of change. The interactions of system components 
and phased change and transitions are important aspects 
of successful scaling up of health services. The authors 
conclude with a call for more research on the manage- 
ment of health system transition and adaptation to chan- 
ging contexts. 


The long, dynamic, and complex process of health system 
development 

It is important to understand the performance of a 
health system in its broader context. This is especially 
important in countries, like China, which are in the 
midst of a number of rapid and interconnected changes. 
Work on health system development can draw on a sub- 
stantial body of work, which applies the lens of resilience 
thinking and CAS to studies of changes in social- 
ecological systems. Resilience thinking offers a good 
framework for examining the long, dynamic, and com- 
plex process of system change. Folke et al. [5] argue that 
adaptation and transformation are essential for main- 
taining system resilience. They view adaptability as the 
capacity of actors in a system to influence resilience and 
transformability as the capacity to create a fundamen- 
tally new system. They differentiate between two types 
of transformation. Forced transformation happens at a 
scale beyond the influence of local actors and is imposed 
by external forces. Another type of transformation is the 
deliberately initiated transformational process by people 
involved at multiple scales and can lead to feedback ef- 
fects that conclude in whole system change. 

The literature on adaptive management and transition 
management has roots in CAS theory [5]. Adaptive 
management is concerned with the establishment of a 
continuous learning process that responds to new infor- 
mation by reformulating hypotheses and models, and 
understanding policy implementation as experiments. 
Transition management is concerned with the dynamics 
of structural change of the system. There is a spectrum 
between adaptability and transformability from the lens 
of resilience. Identifying the key transition points and 
understanding the mechanisms of how system transfor- 
mations are initiated, facilitated, and influenced can help 
us understand the health system development process. 

Gell-Mann argues that it is important to differentiate 
between what is adaptive and the outcome of a process 
of adaptation [17], he maintains that the latter can be 
maladaptive due to the influence of selection pressures. 
From a system evolution perspective, it is crucial to 
understand the impact of individual adaptations for sys- 
tem goal achievements, system resilience, and choices of 
system transition [43]. The mal-adaption is similar to 
policy resistance discussed by Tan et al. [14]. This paper 
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applies these concepts to the analysis of the pathway of 
development of China’s NCMS system over three and a 
half decades. It explores what has triggered transitions 
and how adaptation by agents has played an important 
role in driving change. It also explores how the Ministry 
of Health has attempted to maintain system resilience by 
enabling agents to adapt to a changing macroeconomic 
context while acting to ensure that the system main- 
tained its function and achieved agreed policy goals. 


Methods 

This study is a retrospective review of the development 
of the rural health system over three and half decades. 
We decided to focus on NCMS for several reasons. First, 
the authors have already undertaken research on NCMS, 
and several participated in the implementation process 
or served as advisors to government. Second, the au- 
thors have access to key informants who were involved 
in the policy process. Third, the first author is on the 
National Healthcare Reform Expert Committee. This 
committee includes representatives of all relevant gov- 
ernment agencies including the State Development and 
Reform Commission, the Ministries of Health, Finance, 
Civil Affairs (for the poor), Human Resources and Social 
Insurance, and so forth. 

An insider's perspective has strengths and limitations. 
Being part of the change process provides information 
on the “black box” of policy negotiation and decision- 
making, and the core issues being debated. It can also 
provide insights on the thinking of top decision-makers 
when they were in the conference rooms. However, these 
impressions might not be accurate, and they may be influ- 
enced by a number of biases. Realizing the pros and cons 
of the core data sources, this case study selected supple- 
mentary sources of data to minimize the potential risk of 
bias. 

We approached the former Minister of Health whose 
tenure ended in 2003. We asked him to identify key in- 
formants who had gone through the longest period of 
NCMS development. We identified four key informants 
including the former minister, the rural health bureau 
director, the NCMS office director, and the former 
Shan’xi province Director of Health Bureau. We asked 
the four key informants to define stages of the develop- 
ment of NCMS from their own perspective, and identify 
the key “transition points” for system change. The four 
informants provided similar answers which were based 
on the changes of the national agenda for development. 
Based on the key informants, we built a detailed picture 
of NCMS system development stages and transitions. 

The key informant interviews pointed to the close re- 
lationship between health system change and the na- 
tional development process, and national development 
priorities. ‘'o use the language of CAS, the context plays 
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a key role in driving system transitions. To test this hy- 
pothesis, we began to collect and review all national pol- 
icy documents on CMS and NCMS. China has issued 92 
national policies addressing rural health systems. To 
better understand the national development priorities, 
we reviewed the memoirs of the former premier Zhu 
Rongji, and interviewed the former health minister on 
his reflections of the national policy priorities. 

We paid particular attention to the policy environment 
and public policy priorities at each “transition point” to 
understand the interplay between health policy and na- 
tional development agenda transitions. We developed a 
semi-structured questionnaire to conduct a second 
round of interviews of the key informants on the pro- 
cesses of multi-agency bargaining and negotiation, and 
to identify the key determinants for making the “transi- 
tion” happen. The data on adaptation mainly came from 
three sources. First, we interviewed the NCMS office dir- 
ector, two provincial (Shan’xi and Hubei) health bureau 
directors, and three county health bureau directors to 
learn the process of the NCMS policy formations and 
implementation. We also reviewed internal reports on 
NCMS, collected statistics of rural health system ele- 
ments based on the government health information sys- 
tem, and the policies issued by the national government 
and the health ministry on NCMS. The third source is 
the large body of published literature on NCMS. All in- 
terviews took place between January 2013 and January 
2014. Finally, to understand the status changes of 
NCMS, we traced the policy documents on rural China 
development. Each January, the State Council of China 
issues the Ist National Policy Document, and historic- 
ally, all Ist Documents concern rural development, in- 
cluding health, education, social security, agriculture, 
etc. We hoped that through this method, we would have 
a full picture of China’s NCMS development. 

{t is important to keep in mind the relatively narrow 
focus of our data collection on the perspectives of health 
sector policy actors. We have not attempted to collect 
information on the perspectives of providers or users of 
health services. Nor have we explored the points of view 
of senior policy actors outside the health sector. Despite 
these limitations, the study provides a useful insight into 
the challenges of managing the adaptation of a health 
system to a rapidly changing context, as seen by those 
most directly responsible for providing policy leadership. 


Results 

Starting in the late 1970s, rural health finance went 
through three main transitions. Each was a forced transi- 
tion [5], because it was largely determined by a shift in 
the national development agenda and in the understand- 
ing by the top leaders’ of the key issues concerning rural 
health system development. The adaptation processes 
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after each transition were strongly influenced by an iter- 
ation between responses by a multitude of individual 
agents and government efforts to revise the rules of the 
game, through a series of policy initiatives. Table 1 sum- 
marizes the transition and adaption process of NMCS. 
Detailed analysis follows. 


First transition and adaption to outside pressures 
between 1978 and 2002 

The initial forced transition (from the planned economy 
to the market economy) took place in the late 1970s. 
Launched in late 1978, China’s economic reforms pro- 
moted a system of household responsibility. Land that 
had previously been collectively owned was allocated to 
individual households. The introduction of the house- 
hold responsibility system was a significant contributor 
to the collapse of CMS, since townships could no longer 
allocate a share of collective production to the scheme 
[44]. In addition, low levels of public funding of rural 
health, design, and management flaws of CMS, and the 
lack of consensus about the future of CMS accelerat- 


Table 1 Transition and adaption of NCMS 
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ed disintegration of the program. In the 1980s, CMS 
coverage fell to less than 10% of rural residents, with the 
lowest rate at 5% [2,45,46]. At the same time, the gov- 
ernment ended its policy of requiring skilled health 
workers to remain in rural facilities. Over time, the 
Ministry of Health became aware of a number of prob- 
lems with rural health services, such as rising costs of 
medical care, shortages of skilled personnel, and the re- 
surgence of previously eradicated or controlled infec- 
tious diseases [44,47]. 

The priorities of the Ministry of Health during this 
period were to maintain the effectiveness of the health 
system in meeting agreed policy goals with regard to ac- 
cess to services, while ensuring that health facilities 
remained financially viable. It pursued the latter by 
allowing service providers to use “market tools” to gen- 
erate revenue to pay their workforce [48]. Between 1978 
and 1989, the policies issued by the Ministry of Health 
primarily focused on maintaining the financial viability 
of public health organizations and ensuring that health 
workers were paid. From the end of 1978 to the summer 


Notes 


Time period Events 
1950s to 1978 The origin and development of the Cooperative Medical Scheme (CMS) 
Dec. 1978 China’s market oriented economic reform promoted a system of household responsibility 


1978 to 1979 


The Ministry of Health (MoH) issued five national policies to strengthen 


CMS: System History 
Transition Point | 


Adaption Process | 


rural health care facilities and organizations in staff compensation, training, 
management, investment, and subsidies to health facilities 


1980s 
economy in rural China and other reasons 


The CMS collapsed rapidly due to lack of support from the collective 


The MoH issued many documents on rural healthcare workforce and their compensations, 
such as retirement and pension calculation, subsidies, private clinic permissions, 

support to barefoot doctors, service fee charges for immunization work done by the 
grassroots clinics and doctors, and service fee charges by sanitation and anti-epidemic stations 


The MoH Started to rebuild CMS, but the efforts failed for lack of consensus between 


different government branches and the limited investment to the rural health system 


Oct. 2002 


A "Decision on Further Strengthening the Public Health Work in Rural Areas" was 


Transition Point II 


issued jointly by the Central Committee of the Communist Party of China (CPC) and 


the State Council in October 19, 2002. The New Coop 


erative Medical Scheme (NCMS) was defined 


as a rural cooperative medical insurance system based on a co-financing system which included 
volunteer contributions from individuals and financial support from central and local government 


2003 to 2005 


NCMS pilots were carried out in approximately 300 counties in order to improve 


Adaption Process II 


the design of reimbursement plans, the management of funding and services , etc. 


2006 A large-scale interim evaluation of the scheme was carried out, which helped inform 
subsequent policy and promote convergence in policy design 


2006 to 2008 
to more than 90% in 2008. At the same time, the 


available beds, and inpatients treated has increase 


County Hospitals 


2009 The Central Committee of the CPC and the State Council jointly endorsed and issued the 


The expansion of NCMS in China. The coverage rate 


of NCMS increased from less than 10% in 2002 
number of registered doctors, assistant doctors, 
d significantly in Township Health Centers and 


Transition Point Ill 


Guidelines on Deepening the Reform of Healthcare System after about three years of intense 


debate and repeated revision 


2009 to present 


Policies on Essential Drugs, County Hospital Reforms, Payment Reforms; Integration of 


Adaption Process || 


NCMS with Urban Health Insurance Systems at local level 
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of 1979, the Ministry of Health issued five national 
policies to strengthen rural health care facilities and or- 
ganizations in staff compensation, training, management, 
investment, and subsidies to health facilities. More pol- 
icy documents were issued between 1979 and 1988 on 
the rural healthcare workforce and their payment, such 
as retirement and pension calculation, subsidies, private 
clinic permissions, support to barefoot doctors, service 
fee charges for immunization work done by the grass- 
roots clinics and doctors, and service fee charges by 
sanitation and anti-epidemic stations. By allowing the 
rural healthcare facilities and doctors to charge fees to 
maintain the delivery of health services, and by allowing 
the health facilities to improve their management ac- 
cording to the market principles and fighting for more 
government investment in rural health, the Ministry of 
Health ensured the viability of the rural health services 
in the face of major financial challenges [48,49]; it did 
this whilst maintaining public ownership of these facilities. 
In addition to the public ownership of the healthcare facil- 
ities and the workforce, the professional organizations, 
such as the Chinese Medical Association and the Chinese 
Doctors’ Association, were all under the management of 
the Ministry of Health and helped the healthcare workers 
to negotiate their income, maintain their social status, and 
influence their behavior. 

A few attempts were made to rebuild CMS but they 
did not result in changes at scale because of conflicts be- 
tween government ministries. On the one hand the Min- 
istry of Health hoped to use premiums to reduce the 
financial pressures on its health facilities and pay higher 
salaries. On the other, the Ministry of Agriculture did 
not want to burden farmers with additional fees. This 
conflict of interest was exacerbated because many rural 
health facilities were employing a significant number of 
largely untrained staff who had begun employment dur- 
ing the Cultural Revolution of the 1970s when training 
colleges were closed, and they needed to secure funding 
for their salaries and pensions. Once this group reached 
retirement age in the 2000s and rural facilities were able 
to recruit graduates of newly established medical colleges, 
there was a higher possibility that increased funding 
would provide benefits to rural residents. In addition, 
there was no clear agreement on the relative roles of dif. 
ferent levels of government in financing health services. 
The limited local capacity of collecting fees and the low 
priority in the national reform agenda also contributed to 
the failed CMS rebuilding in the 1990s [50]. 

As public health care facilities and workers increasingly 
operated in a market, the strong, centralized, and hier- 
archical heath care administrative system played an im- 
portant role in maintaining system resilience. However, as 
the former Rural Health Bureau Director Li Changming 
said “The adaptive behavior of the public healthcare 
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facilities and the health care workers under a market econ- 
omy became increasingly difficult for the Ministry of 
Health to control’. 

There were two co-evolving patterns in the transition 
and adaption process: one was the health facilities and 
health workers who were becoming increasingly sophis- 
ticated participants in the health market often with sup- 
port from the Ministry of Health; and the second was 
the efforts by the same ministry to maintain the basic 
health provision system. As one key informant Li 
Changming said during our interview “we are health 
professionals and health officials, we had to make the 
health facilities function to serve the people”. 


Second transition and adaptation: implementing the 
un-implementable through an experimentation process 
The limited government funding of health care facilities 
led them to become increasingly reliant on charges to 
patients. Rural residents had to pay for treatment and 
medicines themselves, frequently resulting in impover- 
ishment and/or foregoing of necessary treatment. This 
was generating pressure on the government to act. 

The 2000 WHO report “Health Systems: Improving Per- 
formance’, ranking the fairness of financial contribution to 
health systems, ranked China at 188. “This was considered 
a loss of face by top leaders”, said by the former Health 
Minister Zhang Wenkang. In addition, “the top leaders 
were shocked by the rural health conditions and the im- 
poverishment of rural health care’ in China and decided 
to reform the rural health care system in 2002 and 
2003. This was in the context of a change in the coun- 
try’s political leaders and a shift in broad development 
policy in favor of taking active measures to ensure that 
all social groups benefited from China’s rapid economic 
development. 

Faced with the difficulty of simultaneously modifying 
multiple components of the health system, the govern- 
ment decided on the development of NCMS as an entry 
point for reform. The new system was named New CMS 
(NCMS) in reference to the widely held view that the 
CMS had been an important achievement of health Sys- 
tem development during the 1970s. Further, “the many 
efforts to re-establish a social health financing scheme in 
rural areas during the 1990s provided very useful lessons 
for policy makers’, Fu Wei, the NCSM office director 
said. 

The landmark policy “Decision on Further Strengthen- 
ing the Public Health Work in Rural Areas” was issued 
jointly by the Central Committee of the Communist 
Party of China and the State Council on October 19, 
2002 [51]. The “Decision” made clear that rural China 
would establish NCMS, and the NCMS would be co- 
financed by the central and local governments and con- 
tributions from individuals. Central government would 
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contribute 10 RMB, provincial and county governments 
were required to contribute 10 RMB, and individuals’ pre- 
miums were set as 10 RMB. The NCMS was designed to 
cover expenses from catastrophic illness and to be man- 
aged at county level instead of village and township levels. 


The basics for developing a functional health system 
Several factors came together to provide a window of 
opportunity for the rapid development of NCMS. The 
Chinese government had made a political decision to 
alter its development strategy to ensure that all social 
groups benefited from economic growth. One aspect of 
this political change was a decision to make fiscal transfers 
to poor rural counties to support improvements in the 
provision of services. This was a major change in the man- 
agement of public finance. The government was looking 
for “quick wins” to demonstrate its seriousness in meeting 
the needs of the rural poor. 

Meanwhile, the Ministry of Health had supported a 
number of experiments with CMS, an organizational ar- 
rangement that would enable local governments to reim- 
burse people for health costs they had incurred. A 
number of experienced experts were available to support 
the development of a national scheme. As a result, a 
mechanism was ready for translating the new govern- 
ment priority into practice. The government began by 
transferring very small amounts of money on a matching 
fund basis to CMS schemes, which reimbursed patients. 
These schemes were found to be effective in managing 
the money. Since the government wanted to transfer lar- 
ger amounts of money, the existence of this mechanism 
enabled it to earmark the money for health on a matching 
funds basis. NCMS provided a functioning mechanism to 
transfer relatively large amounts of money with the confi- 
dence that a substantial proportion of the money would 
be paid to rural people as reimbursement for health care. 
“This was an effective mechanism to ensure large numbers 
of people had visible benefits from this high profile govern- 
ment health program,” Fu Wei said. 

Crucially, in the early 2000s, medical facilities existed 
in rural areas all over China. Many had benefited from 
large government programs of investment in physical 
infrastructure, including health facilities. Also the hier- 
archical health management system was capable of ne- 
gotiating with line ministries and local governments, 
providing a basis for the development of a functioning 
rural health system, indicated by the key informants at 
national and local levels. 

One key aspect of the early phases of policy imple- 
mentation was that pilot counties demonstrated a cap- 
acity to transfer money into a NCM5 fund and to 
ensure that it was used to reimburse patients. The Min- 
istry of Health put a lot of effort into creating an effect- 
ive system for managing these flows of public funds 
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[49]. In doing so, it demonstrated the possibility of sub- 
sidizing services used by residents of relatively poor 
areas [49,52]. 


Experimentation with incentives 

With limited and variable capacity in county health ad- 
ministrations charged with implementing the NCMS, a 
lack of local-level data on burden of disease and health 
service utilization, and reliance on a huge number of 
implementing units, China adopted a experimentation 
process to allow local governments to adapt the scheme 
to local conditions and produce lessons that could con- 
tribute to scheme design and promote bottom-up learning 
in development of a central government policy [53,54]. 

As with many Chinese policies, the central government 
set the parameters within which sub-national govern- 
ments should work. Pilots were carried out in approxi- 
mately 300 counties between 2003 and 2005. Many key 
elements of scheme design were left to local governments, 
including amounts of funding, insurance coverage, and 
design of reimbursement plans, and the management of 
funding and services. Expert teams were convened to help 
guide county-level pilots, develop training materials on 
NCMS design, and carry out training for local government 
officials and NCMS managers. In 2006, a large-scale in- 
terim evaluation of the scheme was carried out. This 
helped inform subsequent policy and promote conver- 
gence in policy design [54]. It was negotiated in the deci- 
sion making period between government agencies, but 
once it was implemented, it became very simple as it was 
primarily the health ministry's job, which injected money 
into the system and the Ministry of Health and health fa- 
cilities started to build the provision again. As Fu Wei said, 
“when funding is available, everyone is satisfied and the 
incentive remains strong’. 


Expanding development investment to other components 
of the health system 

Development of other parts of the rural health system, 
such as the management of medical facilities, drug pro- 
curement, establishment of Monitoring and Evaluation 
systems, and strengthening capacity in rural hospitals 
and clinics, came after the launch of the health insur- 
ance scheme. It is worth mentioning that in the entire 
process of NCMS development, the Ministry of Health 
made significant efforts to negotiate with the Ministry of 
Finance and the National Development and Reform 
Commission (NDRC), which are in charge of health fa- 
cility investment to invest heavily in public health, town- 
ship clinics, and village health stations, as well as county 
hospitals. The Ministry of Health also issued a series of 
policies to manage and improve the rural health work- 
force. The information system is also prioritized in the 
process of development [52]. 
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Changing the rules of the game and the compromise of 
system goals 

The NCMS was initially a win-win for health facilities/ 
health workers and the general population. Rural residents 
received tangible amounts of money as reimbursement for 
medical care. Health facilities generated income from the 
additional demand for services. Poor counties received a 
substantial amount of national and provincial money 
which more than matched their contribution. Over time, 
the facilities adapted to increase their share of resources. 
Further, there was competition between different facility 
levels, so each tried to benefit. Inevitably, this led to cost 
increases. The government responded with efforts to re- 
form the health system and to ensure that a substantial 
share of the benefits went to the general public. However, 
it ran into major stakeholder interests, although the 
NCMS reform changed the rules of the game by providing 
substantial amounts of public finance and asking pro- 
viders to think about the system goals*. 

The evaluation of the NCMS came with the conclu- 
sion that the reform had achieved some success in finan- 
cial protection of the catastrophic health care expenses, 
and the reform was retained [55-57]. The government 
rapidly increased the amount of money it contributed to 
these schemes. 


Third transition and adaption: rural health system under 
the National Healthcare Reform Agenda 
In 2009, China embarked on an ambitious healthcare re- 
form, with the goal of providing affordable and equitable 
basic health care for all by 2020, through universal health 
insurance coverage, establishing an essential drugs system, 
improving the primary health care delivery system, man- 
aging referrals to special care and hospitals, expanding 
public health services, and reforming public hospitals {58}. 
The rural health care reform decision body was located in 
the NDRC with the Ministry of Health? as a member. 
Reform and management of the rural health system be- 
came a part of the national health care reform agenda. 
NCMS-led rural health system development faced a 
complex management and policy environment. N owadays, 
the county hospitals are public but the investment deci- 
sions, financial power, and personnel managements are 
dispersed between many line ministries. The Ministry of 
Health and the NDRC are responsible for investment; the 
Ministry of Health, the Ministry of Finance, the Ministry 
of Human Resources and Social Security, and the Ministry 
of Civil Affairs are responsible for financial power; the 
Ministry of Health, the Ministry of Human Resources and 
Social Security, and the Party's Organizational Department 
are in charge of personnel management [56]. 

After three years’ of reform effort the complexity of 
the change process has become increasingly clear. The 
government introduced an essential drugs policy to 
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control costs and this was met with strong resistance 
from the doctors and the health care facilities [41]; the 
reform of county hospitals faced big challenges [59], and 
the implementation of reform is now understood to be a 
complex and challenging process [56]. While outside 
forces were exerting pressure on the delivery system 
(through financing system-payment reforms, through 
reforms in public county hospitals, through drug selling 
controls, through the creation of a competitive health 
providers’ game, etc.), agents actively adapted to each 
change in the rules to protect their interests. There is 
also intense stakeholder lobbying concerning any changes 
in the roles. For example, the Ministry of Finance has 
had difficulty in pushing through a reform agenda to 
lower costs, increase access to quality care when the 
health system stakeholders are adamant to protect their 
own interests*. 

Because the national health care reform has been slow 
to show signs of success, the 2012 National Economic 
Conference did not include health care reform in its pri- 
ority list [60]. In March 2013, the National Healthcare 
Reform Office was relocated to the newly formed Na- 
tional Health and Family Planning Commission. This 
has been taken by some analysts to be a sign that the 
health care reform agenda has less priority than before 
[61]. The third transition and adaption process is far 
from complete. 


Discussion 

The case of NCMS provides useful insights into health 
system development in low- and middle-income coun- 
tries. Each transition was triggered by a change in the 
national development agenda. When the macro-policy 
environment changed, its influence on the health system 
could be negative (the case of CMS collapse) or positive 
(the establishment of NCMS). Keeping the health agenda 
high in the development core agenda is essential to en- 
sure that forced transitions result in beneficial health 
system development. 

China's rural health system has shown great resilience 
when facing outside forces because it has a strong, cen- 
tralized, and hierarchical administrative system and the 
healthcare facilities are publicly owned. The Ministry of 
Health played a strong advocacy role in pushing for in- 
vestment in facilities. It also promoted experimentation 
with an effective mechanism for managing fiscal transfer 
earmarked for health. The existence of a strong Ministry 
of Health has been an important force for resilience. 

The Ministry of Health has had to balance its respon- 
sibilities for meeting national health system objectives 
with its role as an advocate for the interests of health 
service providers. It adopted various approaches to cope 
with the impact of the market economic reform by 
allowing health care workers and organizations to charge 
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service fees without fully understanding the long-term im- 
pact of the emerging agent behavior on the system goals. 
These emergent behaviors created policy resistance or 
mal-adaption when the goal oriented new reform policies 
were introduced. In some cases, public ownership of hos- 
pitals and clinics has reinforced the behavior of agents as 
they use their position to reduce competition from private 
providers. This kind of interest group behavior could put 
the system at risk if it cannot deliver the expected per- 
formance to the public and to the government. 

The third insight is the need to adapt implementation 
of policies to local contexts. China’s rural health system 
development between 2003 and 2008 shows that incen- 
tive structure is crucial and the limited and varying sub- 
national capacity need not be a barrier to developing 
functioning systems. Capable and motivated local gov- 
ernment, health care officials, health organizations, and 
doctors are important for effective implementation. The 
national policies must be designed to allow flexible local 
management to get the implementation process started 
and, in the process, manage the convergence of the pol- 
icy design. 

Finally, the study suggests that an awareness of CAS 
concepts for understanding system behavior can provide 
a useful tool for analyzing the likely response to different 
policy interventions. The many problems that have arisen 
with the implementation of the rural health system reform 
policy illustrate the complex, political nature of the man- 
agement of this kind of change. In particular, the case 
study highlights the important role of government in es- 
tablishing clear health system objectives and in providing 
overall leadership for the management of system change. 

China and many other countries are likely to experience 
rapid and interconnected changes for many years to come. 
Their health systems will need to adapt to these changes 
and to their impact on broad development policies. It will 
become increasingly important that the government in- 
crease its capacity to manage this kind of complex change 
process in order to create a resilient health system. 


Endnotes 

“Discussions in national health care reform meetings 
where the chief author is a member. 

’Ministry of Health, renamed National Health and 
Family Planning Commission since March, 2013. 

“Internal discussions participated by the first author in 
the national health care reform meetings. 
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Abstract 


Background: Health systems strengthening is becoming a key component of development agendas for low-income 
countries worldwide. Systems thinking emphasizes the role of diverse stakeholders in designing solutions to system 
problems, including sustainability. The objective of this paper is to compare the definition and use of sustainability 
indicators developed through the Sustainability Analysis Process in two rehabilitation sectors, one in Nepal and one in 


Somaliland, and analyse the contextual factors (including the characteristics of system stakeholder networks) influencing 
the use of sustainability data. 


Methods: Using the Sustainability Analysis Process, participants collectively clarified the boundaries of their respective 
systems, defined sustainability, and identified sustainability indicators. Baseline indicator data was gathered, where 
possible, and then researched again 2 years later. As part of the exercise, system stakeholder networks were mapped 

at baseline and at the 2-year follow-up. We compared stakeholder networks and interrelationships with baseline and 
2-year progress toward self-defined sustainability goals. Using in-depth interviews and observations, additional contextual 
factors affecting the use of sustainability data were identified. 


Results: Differences in the selection of sustainability indicators selected by local stakeholders from Nepal and Somaliland 
reflected differences in the governance and structure of the present rehabilitation system. At 2 years, differences in the 
structure of social networks were more marked. In Nepal, the system stakeholder network had become more dense 

and decentralized. Financial support by an international organization facilitated advancement toward self-identified 
sustainability goals. In Somaliland, the small, centralised stakeholder network suffered a critical rupture between the 
system's two main information brokers due to competing priorities and withdrawal of international support to one of 
these. Progress toward self-defined sustainability was nil. 


Conclusions: The structure of the rehabilitation system stakeholder network characteristics in Nepal and Somaliland 
evolved over time and helped understand the changing nature of relationships between actors and their capacity to 
work as a system rather than a sum of actors. Creating consensus on a COMMON vision of sustainability requires 
additional system-level interventions such as identification of and support to stakeholders who promote systems thinking 
above individual interests. 


Keywords: Disability, Health systerns, Nepal, Physical rehabilitation, Social network analysis, Somaliland, Sustainability, 
Systems thinking 
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Introduction 

Health systems strengthening is becoming a key component 
of development agendas for low-income countries 
worldwide. As a means to achieve this, systems thinking 
provides perspectives on how health systems can be 
assessed [1], recognizing non-linearity, complexity, 
heterogeneity, uncertainty, and ambiguity of real-world 
settings [1-4]. The 2009 Flagship Report from the Alliance 
for Health Policy and Systems Research proposes “Ten 
Steps to Systems Thinking”, emphasizing the roles of 
diverse stakeholders in designing solutions to system 
problems, including sustainability [1]. Studying information 
flow mechanisms between actors and within networks can 
help us to understand decision-making processes, as 
well as the social processes which influence the resilience of 
socio-ecological systems (including health systems). Asch 
[5] showed that individuals’ decisions in an unpredictable 
world are often based on peers’ opinions and actions. 
Interactions and collaboration between stakeholders 
depend on various social factors, such as trust, conflict 
resolution, and knowledge integration [6], and also on 
circulation of information within social networks [7,8]. 

The structure of social networks influence individual 
actors’ capacity to respond to the needs of the system as 
a whole [9,10]. It follows that understanding system 
stakeholder networks may be important when analysing 
how information on system sustainability can be used by 
the actors of the system to make informed decisions 
[11,12]. However, the structure of social networks 
may only be one amongst other factors contributing to the 
use of data in decisions. Understanding the dynamics of 
systems therefore requires combining several methodolo- 
gies to capture the complexity of health programmes, the 
embeddedness of systems within other systems, and the 
multi-layered governance of health systems [13-15]. 

In this paper, we build on previous work to introduce 
systems thinking among local stakeholders of the physical 
rehabilitation system in Nepal and Somaliland [16]. 
Although sustainability has been at the heart of recent 
international health programmes and policies, the meaning 
of sustainability remains unclear and confusing to most 
public health professionals [17,18]. The current challenges 
for policy-makers and researchers are to translate the 
concept of sustainability into concrete indicators [19], 
which will help policy makers and health service managers 
make public health and management decisions [20]. 
However, in order to be successful, such a process 
should also attend to the political tensions involved 
in “knowledge production” and “norm creation” inherent 
to sustainability planning in any system. We used the 
Sustainability Analysis Process (SAP), a system-oriented 
tool, which encourages participants to arrive at consensus 
about system boundaries, define sustainability, and identify 
measurable indicators for a sustainable system [21]. 
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During this process, the concept of sustainability is 
upheld as normative [22]. The process also avoids decisions 
taken by a limited number of “experts”. This implies that 
those participating in the consensus building process are 
not only acting in their technical expert capacity, but also 
as “political actors” taking normative decisions on what 
aspects to uphold [23-26]. Involving a wide range of diverse 
actors of the health system, including users, of course raises 
practical problems. For example, the imbalance of power 
existing between different groups of stakeholders [27,28] 
means that some topics can be neglected during this 
process because people who defend them do not receive 
enough consideration within the group [29-31]. The 
final “step” of the process additionally includes piloting and 
re-visiting the measurement of sustainability indicators to 
judge their fit outside of a workshop setting. The consensus 
building process needs to capture the tensions between 
“knowledge production” and “norm creation” in a particular 
context. 

Along with convening and observing sustainability 
analysis workshops in each setting, we sought to ana- 
lyse the contextual factors and the characteristics of the 
social networks to identify the influences affecting actors’ 
decisions about using sustainability data or not [32,33]. 

The objective of this paper is to compare the definition 
and use of sustainability indicators developed through the 
SAP in two rehabilitation sectors, one in Nepal and one in 
Somaliland, and analyse the contextual factors (including 
the characteristics of system stakeholder networks) 
influencing the use of sustainability data. 


Methodology 

In order to capture social phenomena such as management 
decisions and interactions between individuals, an in-depth 
qualitative research approach was adopted. According to 
Fitzpatrick and Boulton ({34] p. 107), qualitative research 
“is used where it is important to understand the meaning 
and interpretation of human social arrangements such as 
hospitals, clinics, forms of management or decision making”. 
In real-life contexts, multiple case study designs are known 
to be appropriate for understanding and interpreting 
complex causal links in natural setting interventions 
[35,36]. We combined three different methods; we used 
stakeholder network analysis, and the SAP at baseline 
(2010) and at a 2-year follow-up (2012). Interviews with 
key informants lent depth to the observations, the analysis 
and helped understand the relationship between the 
structure of the network, the contextual factors, and 
the use (or not) of sustainability indicators. Each of 
these three methods is described below. 


Stakeholder network analysis 
Stakeholder network analysis was used to map key stake- 
holders in the physical rehabilitation system and identify 
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network characteristics. The analysis was conducted in 
both Nepal and Somaliland in 2010 and again in 2012, and 
is detailed elsewhere [37,38]. In summary, stakeholder 
network analysis consists of three stages: (i) describing the 
set of stakeholders in the network/system (using interviews 
with stakeholders and document review), (ii) characterising 
the relationships between stakeholders (interviews), and 
(iii) analysing the structure of the network/system (using 
software, see below) [38]. Stakeholders were defined as 
persons, informal groups of people, or formal organisations 
who may influence the sustainability of the system through 
their interactions and individual or collective actions 
[39-41]. Relationships between actors can be of different 
kinds and depend on various social factors such as trust, 
conflict, or knowledge sharing [42]. However, all these 
social factors are interdependent with one key process: the 
circulation of information between and within social 
networks [43,44]. The second stage of stakeholder network 
analysis consisted of identifying the existence of flows of 
information between actors or, in other words, the demand 
(receiving information) and supply (providing information) 
of information between individuals. This information 
was collected through interviews. Data collected through 
interviews were recorded in an information flow matrix: 
one matrix on the demand for information and a second 
one on the supply of information. Each respondent thus 
generated a row of “ones” and “zeros” for each of the two 
network relations (demand and supply of information): 
“one” symbolising the existence of demand/supply of 
information and “zero” signifying no information flow 
between the two actors. The final matrix was then analysed 
with the software UCINET to generate statistics about the 
network structure (Table 1) [39,45], to visually represent 
relationships within the network and to identify network 
brokers, who control the flow of information and/or 
resources within the network [46]. 


Sustainability analysis process (SAP) 

The SAP is a participatory method based on systems 
thinking, which combines the Process Analysis Method 
five-step approach [47,48] with a conceptual framework, 


Table 1 Definitions of key network characteristics 
measured a 


Characteristic Definition 


Indicator of centrality of the network as a whole [39]. 
Corresponds to the number of direct ties a stakeholder 
has with any other actor compared to the total number 
of direct ties [39]. The higher the percentage of 
betweenness, the more centralised the network. 


Betweenness 


indicator of network cohesiveness [39]. Defined as the 
number of existing ties divided by the number of possible 
ties between stakeholders. Reported on a scale ranging 
from 0 (no ties at all) to 1 (all actors are connected to 

all others). 


Density 


Page 3 of 11 


the Sustainability Framework, which was applied and tested 
in international health [49,50]. The five components of the 
Sustainability Framework, which were used in our study, 
are: health outcomes, service delivery, organisational 
capacity and viability, community capacity and context 
[50]. The SAP also involves five steps, as follows: i) 
Establish a common understanding of the rehabilitation 
system in the local context; ii) Define system boundaries; 
iii) Develop a common vision of sustainability; iv) Select 
measurable sustainability indicators for the local system; 
v) Collect baseline indicator data [21,51]. 

In both Nepal and Somaliland, the SAP method was 
implemented during a three-day workshop sponsored by 
Handicap International in 2010 with key stakeholders 
involved in the physical rehabilitation system. Participants 
were purposively selected by Handicap International and 
Naspir, the national federation of rehabilitation providers 
in Nepal, and by the two national rehabilitation providers 
in Somaliland. The first list of participants was shared 
with and reviewed by the two investigators (KB and JP) in 
relation with the diversity within the rehabilitation sector. 
The participants included representatives from the 
Ministry of Health and/or Ministry of Social Affairs, re- 
gional health authorities, selected rehabilitation professional 
staff (physical therapists and orthopaedic technicians), 
rehabilitation centre managers, representatives of disabled 
people’s organizations, and representatives of international 
donors and non-governmental organizations (NGOs) 
involved locally. 

Two-year follow-up workshops were held in both 
countries in 2012. All the organisations that were repre- 
sented at the first workshop were represented at the 
follow-up workshop. After the SAP was reviewed, workshop 
participants discussed key events that influenced the 
sustainability of the rehabilitation sector during the 
intervening 2 years. Sustainability indicators were re- 
measured, where possible, and participants reflected on 
progress toward self-defined sustainability goals. 


In-depth interviews and observations 

Analytic narrative was used to provide explanations of 
unique events and outcomes, and can serve the interests 
of the social researchers who try to describe what events 
take place, why, as well as their significance to actors 
within a system [52]. Analytic narrative is considered 
“a useful tool for assessing causality in situations 
where temporal sequencing, particular events, and path 
dependence must be taken into account” ([53] p. 1,164). 
The analytic narrative approach consists of interviewing 
key actors and understanding their goals, and the main 
factors influencing their behaviour and decisions [52]. It 
also requires analysis of the interactions between actors 
and their impact on institutional settings: “The emphasis is 
on identifying the reasons for the shift from an institutional 
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equilibrium at one point in time to a different institutional 
equilibrium at a different point in time” ( [54], p. 11). 

Information was collected from key informants on 
experiences with collecting and analysing sustainability 
information by individual actors/organisations. Interviews 
were conducted in private and participants were assured of 
confidentiality to encourage participants to share potentially 
sensitive issues and insights. Interviews were recorded. 
Transcripts and notes from each interview and group 
discussion were read in their entirety before coding 
line-by-line to identify and label ideas and meanings 
conveyed in each small section of text. These codes 
were then grouped and labelled to reflect broader 
themes within the data. Further additions and revisions to 
the coding framework were made on a continual basis as 
higher level constructs were generated, through reviewing 
emerging themes and interpreting them in relation to 
stakeholder network analysis findings. 


Results 

The 2010 (baseline) physical rehabilitation stakeholder 
networks in Nepal and Somaliland 

The structure and properties of the physical rehabilitation 
stakeholder networks in Nepal and Somaliland are 
described and compared elsewhere [16]. Key notions 
are summarised as follows. 

In 2010, the social network of rehabilitation actors in 
Nepal (56 actors) was over twice as large as the network 
in Somaliland (22 actors) and there were substantial 
differences in the types of actors involved in service 
delivery and system governance. Notably, in Nepal, 
three ministries — the Ministry of Health and Population, 
the Ministry of Women, Children and Social Welfare, 
and the Ministry of Peace and Reconstruction — were 
directly involved in governance of physical rehabilitation 
services. In Somaliland, the Ministry of Public Health was 
solely in charge of rehabilitation services, although the 
Ministry of Labour and Social Affairs coordinated broader 
disability issues. 

Country differences in the involvement of local NGOs 
and disabled people’s organisations were also marked. 
In Nepal, local organisations provided services and 
directly managed rehabilitation centres. These organisations 
included professional associations (Nepal Physical Therapy 
Association and Prosthetist and Orthotist Society of Nepal) 
and disabled people's organisations. In Somaliland, a much 
narrower range of actors delivered rehabilitation services. 
Specifically, two non-profit rehabilitation organisations 
(Disability Action Network (DAN), supported by Handicap 
International and the Somaliland Red Crescent Society 
(SRCS), supported by the International Red Cross 
Movement) were responsible for the entirety of rehabilita- 
tion service delivery. Both organisations in Somaliland were 
based in the capital city with complementary networks of 
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partners and providers in the periphery. Disabled people's 
organisations were not involved in service delivery, nor did 
they play a role in advocating for rehabilitation resources. 

In 2010, Somaliland’s system stakeholder network was 
four times more centralised than in Nepal. Nepal’s network 
density was twice as great as in Somaliland (0.2 in Nepal 
compared to 0.1 in Somaliland) (Figures 1 and 2). The 
density of a network is the proportion of all possible ties 
between actors that are actually present. A centralised 
stakeholder network, such as in Somaliland, is thought to 
facilitate communication and innovation, as only a limited 
number of key actors are involved [39,55]. On the other 
hand, centralized networks can easily generate bottlenecks 
if any of the key stakeholders (brokers) block diffusion of 
information and/or resources. In a dense network, such as 
in Nepal, the circulation of information between actors is 
also rapid but with a much lower risk of bottlenecks. 


The 2010 (baseline) self-defined sustainability indicators 
in Nepal and Somaliland 

Discussions that took place during the 2010 SAP workshops 
differed between the two countries. A comparison of 10 key 
self-defined sustainability indicators in each of the two 
countries is provided in Table 2 (the full list of indicators 
chosen is available in [56,57]). In Somaliland, discussions 
about indicators, their measurement, and recommendations 
for the system focused on the two rehabilitation facilities 
existing in that country. Being at the centre of the network, 
the sustainability of these organisations highly affected the 
sustainability of the overall system. Sources of instability in 
the system consisted of the lack of long-term financial 
resources for centres in the capital as well as poor access to 
service users outside the capital. In Nepal, a number of 
rehabilitation centres existed, but populations living in 
remote areas were not reached with the level of resources 
and investment at that time. Sustainability indicator 
discussions in Nepal therefore focused largely on 
mainstreaming physical rehabilitation into priority health 
and social programs, while also transferring some services 
to other actors at the community level. 

In terms of physical rehabilitation goals, network 
members in both Somaliland and Nepal focused on 
increased access to and coverage of rehabilitation services 
across regions (e.g., percent of people with disabilities by 
rehabilitation centre whose physical rehabilitation needs 
have been met). In Somaliland, the rehabilitation providers 
were concerned about the centralisation of services in 
the capital and the provinces being underserved due 
to insecurity outside Hargeisa, the capital, restricting both 
expansion of service provision and the ability of patients 
to complete referrals nade from the periphery. 

In terms of service provision, both stakeholder 
groups recognized that limited numbers and poor 
geographic distribution of rehabilitation professionals 
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Figure 1 The physical rehabilitation stakeholder network of Nepal in 2010 (baseline). 


would threaten the sustainability of their system. In 
Somaliland, network members emphasized the need 
for professionals to provide services outside the capital 
city by analysing human resources regionally. In Nepal, 
stakeholders took a more systemic view and identified the 
total number of rehabilitation professionals who needed 
to be trained in the country over the next few years as a 
sustainability indicator. 

In terms of organisational and financial capacity, 
Somaliland network stakeholders placed strong emphasis 
on the financial autonomy of rehabilitation service 
providers (e.g., number of different sources of funding) 
and team management (e.g., percent of staff with job 
descriptions or number of coordination meeting per year). 
One of the two service providers in Somaliland was 
approaching the end of a funding cycle and had no 
certainty that international support would continue 
after the end of 2011. Hence, sustainability indicators 
specified diverse funding sources and reduced financial 
risk. Some actors identified the emergence of donors in the 
domestic and diaspora private sector as an attractive means 
to diversification, particularly as trust in the political will of 
the Ministry of Health (a potential influential actor within 
the network) to intervene in rehabilitation was low. 

In Nepal, organisational and financial sustainability goals 
are aimed at integrating physical rehabilitation into national 
policies and other programmes (e.g., disability integrated 
into female community health volunteer activities). 
Mainstreaming physical rehabilitation into other social 
or health systems represented an opportunity to secure 
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resources that were not available in an isolated rehabilita- 
tion system. Furthermore, network members in Nepal 
recognised the need to associate with actors outside 
their system working at the community level to increase 
coverage of services. 

In terms of community capacity, the choice of 
sustainability indicators in both countries reflected the 
level of cohesiveness between the rehabilitation services 
and community-based organisations. In Somaliland, 
network members defined “community” as the users of 
the rehabilitation services, and community participation 
was described in terms of the financial capacity of users to 
pay for services (e.g., percent of people with disabilities 
who contributed to the cost of the service) or partici- 
pation of users in the planning of rehabilitation services 
(percent of assessment and planning exercises involving 
community members — i.e., people with disabilities, see 
full indicator list). 

In Nepal, the “community” was defined as the population 
living in areas served by rehabilitation centres rather than 
only existing or potential service users per se. Community 
capacity was perceived as the capacity of community 
organisations to organise themselves (e.g., percent of 
disabled people organisations that have action plans) and 
integrate disability and rehabilitation into their activities at 
decentralised levels (e.g., percent of districts with District 
Disability Rehabilitation Committees and Village Disability 
Rehabilitation Committee, see full indicator list). 

In terms of the enabling environment, in both countries, 
workshop participants recognised the importance of 
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Figure 2 The physical rehabilitation stakeholder network in Somaliland in 2010 (baseline). 


securing political commitment at the national level to 
develop and implement disability-related policies (e.g., 
in Nepal, the existence of a national action plan on 
rehabilitation) and allocate public financial resources to 
the rehabilitation sector (e.g., in Somaliland, percent of 
price covered by Government). 


Physical rehabilitation stakeholder networks in Nepal and 
Somaliland at 2-year follow-up 

The system stakeholder network in Nepal became 
slightly less decentralised but denser over the 2 years 
from 2010 to 2012 with the appearance of seven new 
actors (including international and local organisations) 
(Figure 3). 

In Somaliland, the major change was the disappearance 
of the relationship between the two principle brokers of 
the network (DAN and SRCS). Between the two SAP 
workshops, the two service providers never formally met 
to discuss the management of rehabilitation services or 
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the governance of the system. The system became divided 
into two sub-systems with each service provider at the 
centre and surrounded by collaborative organisations. In 
spite of this system fracture, in 2012, the stakeholder 
network remained highly centralised (61%) and low in 
density (0.1) (Figure 4). 


Comparative 2-year sustainability progress and use of 
self-defined sustainability indicators 

At 2-year follow-up there was little progress towards 
the local vision of sustainability and almost no use of 
the self-defined sustainability indicators in Somaliland. 
System stakeholders individually had not attempted to 
undertake organisational measurements or use this type 
of information in decision making, although most still per- 
ceived the indicators to reflect their vision of sustainability 
of the system. During this period, no organisation took 
the lead to coordinate follow-up of the SAP. Handicap 
International had sponsored the workshop and was 
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Table 2 Example of 10 key self-defined sustainability indicators for the 


and Nepal, by sustainability component 

Sustainability Sustainability indicators 
ponents Somaliland 
Rehabilitation outcomes i 7 


or outputs the centre and whose needs were fulfilled 
Percent of PWDs who were referred to other services 
Service provision Number of physiotherapists and prosthetics and orthotics 


technicians (P&O) in Somaliland 


Percent of regions with at least 2 PT Assistants 


and 2 P&O Assistants 


Organisational and 


Number of different sources of funding 
financial capacity 


Percent of staff with job descriptions 


Community capacity Percent of PWDs and carers who know the 


existence of rehabilitation centres 


Percent of assessment and planning exercises 
involving service users 


Enabling environment — United Nations Level of Security 


Percent of costs covered by Government 


Percent of people with disabilities (PWDs) entering a 
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physical rehabilitation system in Somaliland 


Nepal 
Number of treatment sessions delivered per month 


Number of prostheses and/or theses produced every year 


Percent of centres of who have at least 1 category | P&O 
Number of CAT | P&O who need to be trained by 2015 


Percentage of catchment districts referring patients to centres 


Number of female community health volunteers trained in 
identification of disabilities 


Percent of districts with District Disability Rehabilitation 
Committees and Village Disability Rehabilitation Committee 


Percent of Disabled People Organisations that have action 
plans 


Percent of funding allocated by Government to 
rehabilitation 


Existence of a national action plan on rehabilitation 


perceived as the “owner” of the initiative by some actors, 
who expected the international NGO to continue the lead 
role. Handicap International, however, was reluctant to 
maintain active involvement in coordination since their 
overall objective was to hand-over their support to rehabili- 
tation services to local partners. The Ministry of Public 
Health had limited capacity and political will to fulfil 
this role. The two main service providers continued to be 
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Figure 3 The physical rehabilitation stakeholder network of Nepal in ore ve 


focused on the survival and viability of their own individual 
organisations. Several system stakeholders characterised 
sustainability as “unachievable” under current conditions in 
Somaliland, without sufficient funding and support from 
the Ministry of Public Health. In Somaliland, by 2012, apart 
from a tax break on land for rehabilitation facilities, there 
was still no government funding allocated to provision of 
physical rehabilitation services. 
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Figure 4 The physical rehabilitation stakeholder network of Somaliland in 2012 (follow-up). 


In Nepal, all the sustainability indicators had been 
measured. A couple of months after the first workshop, 
Handicap International offered to take the lead and desig- 
nate one of their project coordinators as the sustainability 
officer in charge of leading the sector on these issues. 
Through the initiative of this international organisation, an 
independent national consultant was commissioned in 
2011 to collect further data to verify the estimates provided 
during the first workshop. During this assessment, 10 
indicators could not be measured because of unclear 
definitions (ie, no clear documentation on what the 
numerator and denominator were) and/or lack of existing 
data collection mechanisms. For example, data on the 
number of districts that have elaborated an action 
plan on disability is not routinely collected in information 
services primarily set-up to measure health service and 
human resource performance. Accessing this type of 
information would require comprehensively surveying 
district health offices and organisations, probably via 
field visits, which was beyond the financial resources 
available for the measurement exercise conducted in 2011. 
Data on the remaining indicators were published in a 
report circulated to the main actors of the rehabilitation 
sector (i.e., Ministry Officers, providers of rehabilitation 
services, and coordinators of rehabilitation programmes 
amongst international organisations). 

Although it is difficult to attribute the precise 
causes involved, evidence from qualitative discussions 
with stakeholders suggested that the SAP, through this 
process of discussion, consensus-building, data collection, 
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and information-sharing, appeared to support the 
advancement of system sustainability in Nepal by encour- 
aging the participation of major stakeholders outside the 
rehabilitation sector in sustainability initiatives between 
2010 and 2012. Following the first workshop and lobbying 
from the main actors of the network, three Nepalese 
Ministries (whose representatives had been invited to 
the final presentation of the sustainability indicators) 
agreed to invest funds to improve the functioning of 
the rehabilitation system as a whole instead of targeting 
specific rehabilitation centres. Their initial idea of creating 
new rehabilitation centres in the country was changed 
after the rehabilitation actors presented their vision of the 
sustainability of the sector. As a result, the Ministries 
agreed to support existing facilities and initiatives. 

In contrast, in Somaliland, systems thinking and action 
was observed during this period only in an area peripheral 
to rehabilitation service delivery, on disability mainstream- 
ing initiatives to increase government involvement in wider 
disability programming in the social sector [58]. This 
involved extensive collaborative work with actors across the 
rehabilitation system, under the auspices of the Ministry of 
Labour and Social Affairs. Partly, motivations for doing 
this work appeared to be in response to specific funded 
opportunities through government and _ international 
organisation channels. These, however, required a far lesser 
commitment of financial resources than opportunities for 
reform of rehabilitation services would have. There was also 
little evidence to suggest that information created during 
the SAP was used in these mainstreaming initiatives. While 
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potentially laying the groundwork for systemic work on 
physical rehabilitation services in the long term, in the 
shorter term, over which this study was conducted, we 
found little evidence of systemic thinking helping to resolve 
the problems rehabilitation actors described facing to 
achieve their vision of sustainability in Somaliland. 


Discussion 

In this study, we compared the selection and use of 
self-defined sustainability indicators in two countries 
to analyse the influence of contextual factors and 
social network structure on the development of physical 
rehabilitation systems. Our assumption was that baseline 
differences between the two networks as well as the 
nature of relationships between actors would influence 
the way the sustainability indicators would be defined and 
used during follow-up. 

The definition of key sustainability indicators was 
implicitly influenced by network characteristics and 
actors’ perception of their own system. In Nepal, the 
stakeholder network at baseline was decentralized and 
dense with a wide diversity of stakeholder types. Here, 
vision of the future system was inherently systemic, 
including concerns about the coverage of services but 
also how the actors of the system work together and 
how integration of new actors could increase system 
impact. The actors of the rehabilitation sector who 
were interviewed recognised the importance of creating 
unity between all the actors of the network in order 
to more effectively negotiate with national authorities 
and donors. Hence, a national body representative of 
all the rehabilitation providers (NASPIR) was created 
in Nepal. In Somaliland, where the stakeholder network 
was centralized in the capital city and low in density 
(few stakeholders and brokers), the vision of the future 
physical rehabilitation system was constructed around the 
two rehabilitation facilities, positioned at the very centre 
of the system, and mainly represented their perspective. 
They focused on the extension of the services towards the 
provinces. 

Social network analysis provides tools to identify 
knowledge brokers, i.e., individuals who create links 
between different groups within a system, such as between 
users and researchers, which was the case of the two 
rehabilitation centres in Somaliland [45]. The brokers in a 
health system also help coordinate actors in times of crises 
or shocks [59]. Other actors essential to the diffusion of 
innovations, such as opinion leaders, champions, or 
change agents, can variously be identified through the 
number of links they have with their peers or non-peer 
actors at different levels of the health system [60,61]. One 
assumption from social network analysis is that the 
position of actors in a network determines their capacity 
to access and diffuse knowledge and information of, In 
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other words, control the flow of information [62,63]. A 
network with a central structure, such as Somaliland’s, 
has more capacity to coordinate actors and provide a 
rapid response, which may be very important during hu- 
manitarian crises [64]. However, in Somaliland, during the 
2-year post-conflict period that we observed, the central 
position of the brokers in this much centralised network 
blocked the circulation of information and the use of 
sustainability data. Promising developments within the 
wider disability social movement in the country may, 
however, help overcome some of these circulation 
blocks between brokers in the future if more actors 
are brought into the network, thereby de-centralising 
information and decision making in the system. 

The use of self-defined sustainability indicators by the 
system was also influenced by individual actors’ survival 
strategies. At 2-year follow-up, the dense network in Nepal 
became even stronger as actors prioritised integration of 
services for organisational growth and survival and was an 
enabling factor in the utilisation of sustainability indicators 
[58]. The emergence of a local champion, the sustainability 
coordinator, who was granted legitimacy by professional 
organisations and financial support by an international 
organisation, facilitated communication necessary to 
continue system sustainability work in Nepal within the 
rehabilitation sector. On the other hand, in Somaliland, the 
changing nature of relationships between the two 
main brokers of the networks completely disrupted 
the circulation of information between actors due to the 
highly centralized, low-density structure of the rehabilita- 
tion sector. This resulted in no follow-up activities to 
monitor or use the self-defined indicators. The space and 
time horizons [11,32], which Somaliland stakeholders used 
to think about sustainability, shrank dramatically between 
2010 and 2012 due to the interruption of international 
funding in the country. In formal network analysis terms, 
the “relationship” between the two main actors of the 
system disappeared in 2012 after they realised that 
their main and pressing priority was the survival of 
their own organisations. Midgley [65] showed that the 
decisions of individuals are primarily influenced by their 
survival instinct. Even well-documented evidence-based 
data cannot influence the decisions of an individual if the 
decision in conformity with the evidence represents a threat 
to his/her own interests and survival (e.g., professional 
career, family situation, or life threatening situation) [66]. 
When brokers shrank their sphere of intervention from 
the system to an organisation, the centralised stakeholder 
network in Somaliland, by nature of the relationships 
between brokers, suffered a bottleneck and therefore 
a barrier to systems thinking. 

As this study demonstrates, adopting a system thinking 
approach involves at least three elements. First, it is essen- 
tial to understand the choices and decisions being made by 


: 


[2S-\~! 
[S293 


ATYARC ING THE APPLICATION OF SYSTEMS THINKING IN MEAITH 151 


¢ 


Blanchet et al, Health Research Policy and Systems 2014, 12:46 
http://www.health-policy-systems.com/content/ 12/1/46 


individual actors; second, to understand the positions 
of actors within the system, recognising that the choices 
of some actors have disproportionate influence on 
the system as a whole; third, it is important to 
understand the wider context affecting changes in 
the system over time (i.e., the existing social networks and 
the relationships between actors), recognising that systems 
are dynamic, social entities which are in constant 
mutation or adaptation [67]. 


Conclusions 

The highly centralised structure of the social network in 
Somaliland had potential to help rapidly diffuse information 
between actors, which might be very useful in contexts of 
emergency (conflict or natural disaster). However, the 
rupture of relationship between the two central actors of 
the network completely disrupted the functioning of the 
rehabilitation sector and lead to the non-use of sustainabil- 
ity data in a sector that became the sum of dispersed actors. 
In Nepal, the cohesion between actors was maintained 
thanks to the role of a local champion and the injection of 
additional funds in the sector. The network remained very 
dense and decentralised and actors there appear to be 
gradually building a systemic vision of their sector, which 
takes account of data such as sustainability indicators for 
planning and negotiation purposes. These findings suggest 
that using sustainability indicators for a health system 
requires cohesion within the system between all (or most) 
actors, as well as an understanding, by actors, of the benefit 
of a collective vision for the sector. Contextual factors, such 
as the availability of funding for activities that primarily 
benefit the system rather than individual actors or organisa- 
tions, can also support this. Further research is needed to 
analyse the different strategies that are required for health 
system interventions to alter the characteristics of social 
networks in social contexts for a collective good. 
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Abstract 


Background: Starting in 1999, Concern Worldwide Inc. (Concern) worked with two Bangladeshi municipal health 
departments to support delivery of maternal and child health preventive services. A mid-term evaluation identified 
sustainability challenges. Concern relied on systems thinking implicitly to re-prioritize sustainability, but stakeholders 
also required a method, an explicit set of processes, to guide their decisions and choices during and after the project. 


Methods: Concern chose the Sustainability Framework method to generate creative thinking from stakeholders, create a 
common vision, and monitor progress. The Framework is based on participatory and iterative steps: defining (mapping) 
the local system and articulating a long-term vision, describing scenarios for achieving the vision, defining the elements 
of the model, and selecting corresponding indicators, setting and executing an assessment plan,, and repeated 
stakeholder engagement in analysis and decisions . Formal assessments took place up to 5 years post-project (2009). 


Results: Strategic choices for the project were guided by articulating a collective vision for sustainable health, mapping 
the system of actors required to effect and sustain change, and defining different components of analysis. Municipal 
authorities oriented health teams toward equity-oriented service delivery efforts, strengthening of the functionality of 
Ward Health Committees, resource leveraging between municipalities and the Ministry of Health, and mitigation of 
contextual risks. Regular reference to a vision (and set of metrics (population health, organizational and community 
capacity) mitigated political factors. Key structures and processes were maintained following elections and political 
changes. Post-project achievements included the maintenance or improvement 5 years post-project (2009) in 9 of the 

11 health indicator gains realized during the project (1999-2004). Some elements of performance and capacity weakened, 
but reductions in the equity gap achieved during the project were largely maintained post-project. 


Conclusions: Sustainability is dynamic and results from local systems processes, which can be strengthened through 
both implicit and explicit systems thinking steps applied with constancy of purpose. 
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Sustainability, Systems thinking, Urban health 
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Background 
Whether we understand the social world (including 
health systems) as operating through systems, or we take 
systems thinking simply as a useful mental construct to 
deal with the complexities of our social world [1], there 
is a growing attention to systems thinking in multiple 
fields of development practice and research, including 
global health. In the global development literature, one 
may argue that this school of thought started with 
Amartya Sen’s book, Development as Freedom [2], and 
Rihani’s Complex Systems: Theory and Development 
Practice {3|. Studies and implementation guidelines for 
‘systems thinking’ have been published in journal sup- 
plements in 2006 and 2007 [4,5]. The Alliance for 
Health Policy and Systems Research published Systems 
Thinking for Health Systems Strengthening in 2009 [6], 
and then supported the publication of a Journal sup- 
plement in Health Policy and Planning in 2012, dedi- 
cated to the applicability of systems thinking tools for 
health systems strengthening [7]. Over the same period 
(2008 to 2010), different authors have used case studies 
to describe how complexity and adaptation play a cen- 
tral part in capacity building [8-10]. Overlapping with 
the field of global health and global development, the 
world of evaluation is itself trying to better approach 
non-linear realities, complexity, and systems thinking 
[11]. Williams [12] presents 11 evaluation case studies 
reporting the use of systems methods and concepts to 
evaluation, and Patton [13] writes and teaches on 
Developmental Evaluation, a methodology explicitly an- 
chored in an understanding of the world as composed of 
overlapping open systems. Discussions, research, and evalu- 
ation on sustainability in health programs appear to have 
followed a similar evolution (see “A systems understanding 
of sustainability” below). Over the same decade and a half, 
a group of practitioners, working in community health at 
local levels and globally, followed an interesting if not iden- 
tical intellectual trajectory, when dealing with the specific 
issue of improving the sustainability of their interventions. 
Non-Governmental Organizations (NGOs) working glo- 
bally in maternal, reproductive, and child health formed the 
Child Survival Collaborations and Resources Group in 
1997 (now known as the CORE Group’). In 2000, CORE 
partnered with a US Agency for International Development 
(USAID) project implemented by Macro International in a 
study called the ‘Sustainability Initiative in order to im- 
prove the conceptualization and implementation of more 
sustainable strategies in community health [14,15]. By 2002, 
this collaboration had produced a tool (the Sustainability 
Framework) for sustainability planning and evaluation. By 
this time, Concern Worldwide Inc. (Concern) had been 
implementing an urban health project in two Municipalities 
of Northern Bangladesh since 1999. Concern’s interest in 
capacity building and sustainability led to a time of critical 
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questioning at the time of its mid-term evaluation in 2002, 
2 years before the end of the project [16]. Concern needed 
an evaluation and learning tool able to guide the implemen- 
tation of project strategies (by the municipalities them- 
selves) toward greater odds of sustainability. The interest in 
learning, however, bridged the local Bangladeshi and the 
global context when USAID provided additional funds to 
Concern to carry out a post-project evaluation 3 and 5 years 
after the end of the project (2007 and 2009). 


A systems understanding of sustainability 

Conceptual and methodological debates about sustainabil- 
ity continue in the literature [17,18], but the identification 
of complex systems behaviors as fundamental determinants 
of sustainability, which was already identified to some 
extent by a number of past authors [19-21], has become 
more explicit in recent publications [3,9,22-25]. 

Under the Sustainability Framework, sustainability is seen 
as resulting from processes taking place in a local system 
where a wide array of stakeholders share responsibility to 
generate and maintain positive health outcomes for their 
community, inclusive of its most vulnerable groups. It of- 
fers an interactive model for assessing progress on critical 
dimensions [26], such as the health outcomes being 
promoted, characteristics of health services (quality, 
accessibility, equity), institutional capacity and viability 
of local government and civil society agencies with long- 
term responsibility for the outcomes, capacity in beneficiary 
communities (e.g., social capital, community organization, 
knowledge/skills, resource mobilization), and socio- 
ecological conditions enabling the work of these local 
agents. 

This paper presents how Concern adapted and used a 
systems approach to place sustainability at the front end 
of project implementation and learning, to build con- 
sensus, find common values, use data for learning and 
adaptive management, and assess progress toward sus- 
tainability, during and after the life of the project. 


Urban health in Bangladesh and Concern’s urban health 
model in Saidpur and Parbatipur (1999-2004) 
Bangladesh is a low-income country with poor health in- 
dicators. Its under-five mortality decreased rapidly in the 
1990s, then slower in the 2000s [27,28]. The infant mor- 
tality rate decreased from 72/1,000 live births in 2004 to 
57/1,000 in 2007 according to the Demographic and 
Health Survey [29]. The fastest growing sector of the 
population lives in urban areas and a third of those, in 
urban slums. The urban population grew from 23% of 
the total population in 2001 [30] to 28% by 2010 [31]. 
This population is largely vulnerable, impoverished, mal- 
nourished, and receives poor health care services [32]. 
Municipalities are legally tasked with ensuring the de- 
livery of primary health care services to the population 
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but had developed almost no capacity to do so at the on- 
set of Concern’s project. For instance, due to limited re- 
sources, public-sector health services were not able to 
meet the existing needs in 1999. Private health care pro- 
viders were the main source of curative care, including 
tertiary and specialized services to the urban popula- 
tions, but had limited or no interest in providing pre- 
ventative and health promotion services. 

In 1995, the Ministry of Local Government, Rural 
Development and Cooperatives issued a directive for the 
effective implementation of expanded programs on im- 
munizations, along with primary health care and family 
planning services through a coordinated effort involving 
the Ministry of Health and Family Welfare, NGOs, and 
private providers. Committees were recommended, al- 
though not established, at three different levels to ensure 
effective health service delivery: inter-ministerial commit- 
tees, central committees at municipal levels, and Ward 
Health Committees (WHCs) at the community level. 

Concern initiated a USAID-funded child survival pro- 
ject in the municipalities of Saidpur and Parbatipur in 
Nilphamari and Dinajpur districts in 1998, with full im- 
plementation from 1999 to 2004. The two municipalities 
had a direct beneficiary population of 74,000 women of 
reproductive age and children under 5. Concern selected 
what, at the time, was a non-traditional capacity building 
approach, based on a partnership with the two mayor’s 
office and their under-resourced municipal health depart- 
ments (MHDs) [33]. Concern supported the organizational 
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development of the municipality cabinets and through 
them developed the capacity of WHCs at the community 
level. In turn, and collaboratively with the MHDs, the 
WHGs recruited, trained and supported a network of com- 
munity health volunteers (CHVs) and traditional birth at- 
tendants, who carried out community and household-level 
health promotional activities. 

Two years into the project, the results of the mid-term 
evaluation were very promising but pointed out to the 
lack of measurable results and signaled important sus- 
tainability challenges. Concern’s strategic response to the 
mid-term evaluation implicitly relied on systems think- 
ing (Table 1). The project, however, also needed a 
method, an explicit set of processes, to guide a diversity 
of stakeholders in the pursuit of sustainable health goals. 
Concern chose the Sustainability Framework as the tool 
through which it could organize the creative thinking of 
multiple stakeholders, create a common end-goal, and 
monitor progress on its redesigned sustainability strategy. 

HICAP, Health Institution Capacity Assessment; MHD, 
Municipality Health Departments; MOH, Ministry of 
Health; WHC, Ward Health Committee; MOHFW: Minis- 
try of Health and Family Welfare is the same as MOH and 
was removed. 

We now describe how the Sustainability Framework 
method was implemented following the mid-term evalu- 
ation (2002), up through the final evaluation (2004) [34], 
all the way to the 5-year post-project sustainability evalu- 
ation in 2009. The steps of implementation, sustainability 


Table 1 Implicit and explicit operationalization of systems thinking for Concern, Saidpur and Parbatipur 


Principle in systems thinking? 


+ Making the distinction between the national health system and local urban 
administrative systems, and focusing capacity building on the MHDs 


Consider boundaries of system, 
nested and overlapping systems 


Construct reality through 


multiple perspectives Ses 
— - Participatory process 


Value and build on relationships 


- WHC as a hub for health promotion 


- Accountability of the municipal government to communities 


Iterative learning 


* Involvement of MHD and WHCs in Knowledge, Practice and Coverage 
Household Health Surveys including disaggregation by wealth quintiles to 


assess equity 


Concern’s management decision (implicit system approach) 


- Voices from all segments of society included in process 


- Forum creation for relation between MHD — WHC — MOH 


- Defining assessment criteria in HICAP and WHC 


Sustainability Framework (explicit 
method) 


- Local system and stakeholder 
mapping 


- Identification of interrelated roles 
between MOHFW and MHD 


- Visioning and scenario planning 
- Iterative, evidence based, and 
* participatory review of progress 


- Gender and equity awareness building 
through diversity in planning 

- Participatory assessment, bringing 
diverse stakeholders together 


» Mutual accountability through 
review of progress 


- Regular measurement of outcomes in 
multiple dimensions of assessment 


- Cycle of visioning, defining measures, 
measuring, reviewing, adjusting 


cy here is not one set of recognized systems thinking principles, and this table is not exhaustive. Williams [12] emphasizes boundaries, perspectives, and 
relationships as central to systems thinking. Many authors emphasize learning, and specifically iterative learning, as an essential principle. Other authors also stress 
the value of collaboration across disciplines, sectors and organizations, transformational leadership, alternative scenario planning, using diversity of stakeholders 


to brainstorm and design, etc. [6,7,26]. 
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planning, project evaluation, and post-project sustainability 
evaluation are summarized in Figure 1. 


Methods 

The sustainability assessment is an iterative exercise de- 
signed for intervention design, evaluation, and continued 
learning. It requires the definition and planning of a 
multi-dimensional evaluation model and its major com- 
ponents, based on the Sustainability Framework (see 
“Defining sustainability for health interventions in global 
development” below), and measurement steps through 
field investigations, which took place through ongoing 
project monitoring and evaluation until 2004, and after 
the project's end. 


Defining sustainability for health interventions in global 
development 

We define sustainability as resulting from a collective 
process within a local system, which maintains or im- 
proves the health status, or a sub-set of health outcome 
indicators, of the locale’s citizens, particularly its most 
vulnerable members. 

Individuals, community groups and structures, and 
government and civil society organizations constitute a 
local system within a larger environment, and it is ultim- 
ately their coordinated social interactions and efforts, 
based on the understanding of their own health and de- 
velopment, which will lead to lasting health conditions. 

The loss of control over local processes beyond a set 
date is inherent in project approaches. This means that 
the immediate determinants of sustainability are based 
on a local process of negotiation, role definition, and 
action, and are effectively outside the full control of a time- 
bound project. Projects, nonetheless, have an essential re- 
sponsibility in advancing the key determining conditions 
for sustaining outcomes within the local system. 
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The value of the Sustainability Framework relies heavily 
on the quality of its contextual development and imple- 
mentation process. The method is described elsewhere as a 
participatory process involving the six steps described in 
Figure 2, bringing together situation assessment, plan- 
ning, evaluation, and strengthening relationships be- 
tween the actors, based on consistent reference to data 
and learning steps [23]. 

We describe now how these steps were adapted by 
Concern and which elements of planning and evaluation 
were integrated in the model: 


1) Defining (mapping) the local system, and the 
common long-term vision 


In February 2003, Concern and its partners defined 
the system of local actors expected to carry out the task 
of health promotion at the municipality level during a 
6-day workshop using stakeholder mapping, and devel- 
oping a common vision through participatory group 
activities [14]. This initial workshop involved Concern 
project team members, 15 municipality staff nominated 
by the municipalities, and Concern capacity building 
and child survival advisors. All work was conducted in 
Bangla in small groups. Final statements were finalized 
in plenary and presented to municipal and ward institu- 
tion leaders. This provided a safe environment for the 
development of a shared vision and discussion of con- 
textual challenges. The central constituents of the sys- 
tem that was mapped were Mayors, elected Councilors, 
and the MHDs in the leadership role; WHCs as an ex- 
pression of the communities, working through CHVs, 
and local health care providers, such as a local hospital 
and NGO clinics. This system definition encouraged a 
broader inclusion of WHC members (not just leaders) 
and CHV representatives in future exercises (2004, 2007, 
and 2009). 


| | 1999 | 2000 | 2001 | 2002 


POOL PPEADOROD PP ERARROEPD bree nonbneverines MineHPPIDPEDDPPOEPP OD >PPHDEDIPPPDYPPPDIPEDEPPYDIPALPPPPPPPIODIPIPE 


Full Child Survival Project 
intervention in Saidpur & 
Parbatipur 


Figure 1 Implementation and evaluation phases of the Saidpur and Parbatipur Child Survival Project. 


| Capacity Assessments (HICAP and WHC Capacity Assessment) 


KPC KPC 
Survey Survey 
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Survey Survey 
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Figure 2 Sustainability framework steps of learning. 


‘Sustainable Assessment Steps of Learning __ > Emphasis | j 


CO MMP 


2) Describe scenarios to achieve the vision, define the 
elements of the Sustainability Framework and their 
indicators (Steps 2 and 3) 


Although the language of ‘scenarios’ was not promin- 
ent at the time of Concern’s initial planning efforts, the 
project helped partners envision not only a workable fu- 
ture but also rational roles for all parties in order to en- 
sure population health benefits were sustainable by local 
stakeholders. The vision and unfolding strategy were de- 
signed to integrate equity issues at the onset. Cognizant 
of the care-seeking barriers faced among the poorest 
urban dwellers, Concern sensitized elected leaders at the 
municipality and community levels of the importance of 
including all people irrespective of ethnicity or class in 
health promotion efforts. Leaders were encouraged to 
provide special assistance such as arranging transport, 
seeking support of an absent husband, accompanying 
the client to the health facility, and/or negotiating of fees 
and payments to those in greatest need. 

The Sustainability Framework examines inter-dependent 
components of evaluation considered essential to sustaining 
health outcomes, each component including different ele- 
ments. Following this, in the first step, Concern and the 
municipalities defined which elements of the Sustainability 
Framework fit their situation, and for each they defined 
how measurements or assessments would be carried out. 
Discussion and participation was maximal in defining what 
should be measured and which issues were of importance 
to achieve sustainable health outcomes. However, Concern 


Y AND SYSTEMS 


project leaders and facilitators certainly played a leader- 
ship role in proposing indicators, and ensuring that 
knowledge, practice, and coverage (KPC) indicators, for 
example, aligned with international standards. Defin- 
ition of capacity indicators and statements combined 
different participatory and expert influences, as de- 
scribed below: 


e Health outcomes themselves were assessed through 
repeated small sample, population-based KPC health 
surveys [35,36]. KPC surveys with samples of 350 to 
600 mothers of children 0 to 23 months had been 
carried out at the onset of the project (1999) and 
were repeated at the end of the project (2004), 
and again in 2007 and 2009. Practice and coverage 
indicators provided the hard benchmarks to assess 
success or failure, as they directly reflected benefits 
to the population. During the process, elected 
leaders were challenged about inclusiveness and the 
participation of all community members in health 
promotion efforts, irrespective of ethnicity or class”. 

e Capacity of both municipalities was assessed 
through the Health Institution Capacity Assessment 
Process (HICAP). The HICAP is a participatory, 
organizational self-assessment developed by Concern 
with staff from the municipal health departments 
[34]. The final selection of domains and indicators of 
capacity assessment was carried out by combining 
an appreciative inquiry approach and commonly 
available organizational assessment tools. 
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| e The HICAP was then carried out with cabinet 


members, ward commissioners, and the health 
department of both municipalities in 2004, 2007, 
and 2009. Non-project staff at Concern led assessment 
workshops in Bangla, taking 3 days per municipality. 
The HICAP describes progress towards an “ideal 
capacity”, as defined by the municipality leaders 
through “possibility statements”. These statements 
provide norms of institutional behavior, as scored on a 
five-point scale and based on dialogue and consensus 
among participants. 
The Sustainability Framework considers institutional 
viability as related to but distinct from capacity; 
“Organizational Viability, relates not only to 
financial viability, but also to other essential types of 
support and relationships—connectedness—which an 
organization depends on to fulfill its mission” (16). 
Through collective reflection and analysis, viability 
elements were identified in the model, such as 
Mayor-Ministry of Health collaboration and resource 
leveraging. Given that the project relied heavily on the 
leadership of the elected mayors, councilors, and their 
political party, the discussion of the viability of the 
model made clear that steps were needed to 
“neutralize” the political risks, and to ensure the 
continuity of support beyond the current municipal 
administrations. Local actors involved in the 
sustainability planning phase helped Concern take 
steps ranging from informing political leaders about 
the role of Municipalities in health promotion during 
the elections and promptly orienting successful 
candidates shortly after elections, to helping the WHCs 
to uphold an apolitical identity in their neighborhoods. 
The project and its partners identified the capacity 
of WHCs as the main proxy measure for the 
community. Concern developed the WHC Capacity 
Assessment tool, similar in structure to the HICAP, 
and used by the 24 Saidpur and Parbatipur WHCs 
in June 2004, April to June 2007, and again in 2009 
to assess their own capacity. The WHC Capacity 
Assessment tool was informed by the HICAP 
development process and the national terms of 
reference for the WHCs. The dimensions of 
assessment and indicators were validated during 
stakeholder consultations with the municipal 
cabinets, health departments, and two purposively 
sampled WHCs in each of the municipalities. 
Reviews were conducted in Bangla and Urdu by 
trained facilitators from the project staff (in 2004) 
and then by municipal constituents (in 2007 and 
2009). The 1-day sessions included guided discussions, 
followed by scoring capacity areas on a five-point scale. 
Areas of assessment included not only the WHC'’s 
internal operations but also the inclusiveness of all 


socio-economic groups and efforts extended 
toward vulnerable community members and 
CHVs. Assessment of CHV’s coverage and retention 
began in 2007, but specific measures of their activities 
and qualities of their interventions were unfortunately 
not systematically monitored. 

e The Sustainability Framework further challenged 
implementers to consider socio-economic threats 
that could undermine efforts towards a viable health 
intervention. Recurrent seasonal neighborhood 
flooding and cultural issues were identified as 
significant impediments to the desired collective 
vision. These impediments were addressed through 
providing a clear role to WHCs in coordination 
with emergency response and water and sanitation 
efforts. Additionally, provisions were made to 
increase social support for decision-making when 
a woman and/or child required immediate health care 
in the absence of the husband, and to generate parental 
and community support to allow mostly young and 
female CHVs to fulfill their home visit duties. 


3) Develop and implement the assessment plan (Step 4) 
and engage stakeholders in analysis and decision 
making (Steps 5 and 6) 


Both the final evaluation (2004) and the post-project 
sustainability assessments (2007 and 2009) involved it- 
erative sequences of participatory evaluation steps: 


e Formation of an evaluation team with Concern and 
municipality participants, under the guidance of an 
external lead evaluator. 

e Analysis of surveys (health survey, capacity 
assessment) and available secondary data. 

e Review, framing, and clarification of evaluation 
questions. 

e Interviews of key informants, individually and 
through group discussions, including the mayor, 
cabinet members/WHC Chairs, Health Inspector, 
Municipal Health Staff, past and current CHVs, 
MOH partners, Ministry of Local Government, 
WHCs members, the Municipal Essential Service 
Package Coordinating Committee, and non- 
governmental health sector partners. 

e Participatory review of findings, including a 
discussion on conclusions and next steps. 


Results and discussion 
Post-project achievements 
From the moment Concern and the Municipalities 
took stock of mid-term achievements versus risks for 
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sustainability, a culture of regular consultation and 
learning became central to the life of the project. As 
assessments (including the HICAP and KPC) were 
highly participatory, the findings of the assessments 
made sense to the stakeholders (as suggested by the ac- 
tions taken by stakeholders, and discussions with the 
evaluator). Participation in the selection of indicators 
was, of course, informed by population surveys special- 
ists and technical guidance. However, the role of local 
stakeholders went beyond obtaining authorizations. 
Municipal health departments were part of the design 
team defining which indicators should be measured, 
and were then central to the analysis and discussion of 
the results. 

The overall findings of the 2009 post-project sustain- 
ability evaluation was largely positive: “From 2004 (end 
of the project) to 2007 (first post-project sustainability 
evaluation), in spite of a near total [98%] reduction of 
external inputs, the municipalities were able to maintain 
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basic operations and observed mostly stable values for 
maternal and child health outcomes. From 2007 to 2009 
(last post-project sustainability evaluation), in the ab- 
sence of any further inputs by Concern, basic capacity, 
operations, and health indicators were maintained, but 
municipalities identified critical gaps in the governance 
and strategic guidance of the model, despite weaknesses 
in human resources management and national level in- 
volvement’ [37]. 

Table 2 presents the evolution of 11 indicators com- 
pared to national urban trends during and after the pro- 
ject. We can summarize the table as changes observed 
during and after the life of the project: 


e These 11 coverage indicators showed notable 
improvements during the life of the project. The 
2004 final evaluation reviewed possible confounding 
factors to the attribution of results to the project 
and was supportive of a substantial attribution of 


Table 2 Child Health Indicator Trends in Saidpur and Parbatipur (KPC) and Bangladesh Demographic and Health 


Survey Comparisons (urban or national average) 


Indicator 


Child Health Complete immunization 


Vitamin A supplementation 


Exclusive breastfeeding 


Complementary feeding of children 6 to 


11 months 


Additional feeding and fluids for the sick child 
Additional fluids (only) for the sick child 
Proper child acute respiratory infection 
identification and referral 


Maternal & Neonatal 
Health 


At least one prenatal consultation during last 
pregnancy 


At least three prenatal consultation during last 
pregnancy 


At least one tetanus toxoid dose during last 
pregnancy 
Delivery by skilled attendant 


Delivery in health care facility 


Immediate breastfeeding 
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1999 2004 1999-2004 diff 2007 2009 2004-2009 diff 
(P value) (P value) 

S&P* 44% 91% <0.05 90% 91% NS 
urban 70% 81% 86% 

S&P 9-37 oarareo. -<0.05 71% 79% NS 
urban 76% 85% 90% 

S&P aa oe 9 <005 70% 73% NS 
nat'l 42% 36% 43% 

S&P 46% 64% <0.05 64% 65% NS 
nat'| 

S&P 25% ~=44% <0.05 34% 25% <0.05 
nat'l 50% 52% 38% 

S&P 24% ~=— 334% ~— <0.05 66% 52% <0.05 
urban 48% 35% 49% 

S&P =—59% 89% <0.05 87% 95% NS 
nat'l 37% 56% 60% 

S&P 64% 62% 74% <0.05 
nat'l 25% 27% 32% 

S&P = 46% ~=— 89% )~— <0.05 57% 70% <0.05 
urban 88% 88% 84% 

S&P 331% ~=—50% ~=—<0.05 56% 59% <0.05 
natl 22% 13% 18% 

S&P 25% 45% <0.05 49% 57% <0.05 
nat'l 8% 9% 15% 

S&P = 26% )3=— 57% ~— <0.05 50% 64% NS 
urban 23% 22% 41% 
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effect to the project [34]. Impact on equity was only 
taken into account by Concern in a follow-on 
project in seven municipalities. However, coverage 
indicators for Saidpur and Parbatipur in 2004 (end 
of project) among the poorest 20% of surveyed 
households were two to five times that of the 2005 
baseline estimates in the seven neighboring 
municipalities. Given the relative comparability of 
the initial and expansion municipalities, this could 
suggest that impact in Saidpur and Parbatipur 
had been far from negligible among its poorest 
population. 

e By the time of the post-project sustainability 
evaluations (2007 and 2009), the main point of the 
evaluation shifted from the project to providing 
information to the municipalities themselves about 
progress toward their vision of sustainable health. 
Questions of attribution of results to the original 
project itself became less critical. As shown in 
Table 2, the initial improvements in 11 indicators 
of maternal and child health realized during the 
project (1999 to 2004), 9 were maintained or improved 
during the 5 years post-project, even though external 
funding dropped to almost zero over this period*. 

In only two cases did an indicator worsen between 
2004 and 2009. 


Self-assessment of capacity at the MHDs, using the 
HICAP tool, progressed substantially from the 2002 to 
2003 baseline assessments to the end of the project 
(2004). By 2007, the scores on the HICAP faced a ceiling 
effect. The assessment had, however, helped guide and 
institutionalize the basic functions and operations re- 
quired of the MHDs to support health promotion in the 
community, including through small amounts of finan- 
cial support to each WHC. 

The 2009 assessment revealed maintenance of the 
structure and basic functions of the WHCs, along with 
weaknesses in their operations. All WHCs had main- 
tained a bank account with a solid balance. They mobi- 
lized additional resources and obtained financial support 
from the municipalities for special events, as well as 
emergencies affecting the poorest members of the com- 
munity. WHCs, nonetheless, expressed dissatisfaction 
with the inconsistent support from municipalities. 

While performance issues at the WHC level and in the 
WHC support of CHVs were identified, the human in- 
frastructure continued to operate at the time of the 
5 year post-project sustainability assessment (2009), des- 
: pite a high degree of national political instability and the 
; food price crisis of 2007 to 2008. The diversification of 
perspectives from diverse actors through the sustainabil- 
| ity assessment phase allowed avoiding the natural bias of 
representation (male, elite) within the WHC and led 
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WHC membership to be more representative of all seg- 
ments of the neighborhood (i.e., class, ethnicity, educa- 
tion level, gender, and political affiliation). Additionally, 
having formed a vision of a desirable public good with a 
diverse group of stakeholders, the project was able to in- 
volve them in informing political party leaderships (both 
in power and opposing) about the role of \WHCs and the 
Councilors. By the time of the post-project sustainability 
assessment, most WHCs had gone through a change of 
Chairpersons following municipal elections, but contin- 
ued operating as largely apolitical institutions dedicated 
to promoting the common good. 


Sustainability needs to be assessed as resulting from a 
local system’s process, not an end-point 

Concern worked with stakeholders and generated a 
process-within-a-system. The level of sustainability 
achieved was a partial achievement. It proceeded from 
also partial but expressed capacity, collaboration, coord- 
ination, occasional cooptation, some loss of energy but 
maintenance of key elements, such as operations of the 
WHCs and CHVs. While efforts of different parties may 
have been imperfect, they aimed to contribute to a rec- 
ognized public good (preventive health outcomes). Es- 
sentially, sustainability occurred as a process supported 
by a network of system agents‘, 

Concern clearly encouraged this by stepping out of 
direct implementation and very rapidly supporting local 
stakeholders in negotiating their long term roles. The 
project not only aligned to a national policy, but since it 
had not been implemented on the ground, it operation- 
alized it and helped local stakeholders in giving it 
substance. 

The fact that both mayors and most elected officials 
participated in the sustainability assessment 5 years after 
the end of activities speaks of the ownership that was 
created. This was built through very intense and persist- 
ent efforts at “accompanying”, or softly leading coun- 
cilors, staff, WHC members, and volunteers in the early 
phases of the project. 

Interestingly, the approach of Concern, supported by a 
planning and evaluation tool for sustainability, can be 
compared to systems thinking design steps highlighted 
in the Alliance for Health Policy and System Research's 
“flagship publication” [6]: 1) The project convened stake- 
holders repeatedly and at every step. 2) Concern led not 
only collective brainstorming but also helped stake- 
holders define the road ahead, the modes of monitoring 
and evaluation, down to the indicators when possible, 
and then jointly review findings (not just with the 
leaders and experts). 3) The definition of elements of the 
Sustainability Framework provided a conceptual map of 
expected results and scenarios for progress. 4) The itera- 
tive steps of assessment, action, and reviews allowed a 
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measure of adaptation at each step. The last post-project 
sustainability evaluation showed, to some extent, how 
this assessment for action principle had been institution- 
alized by the municipality partners. While Concern had 
stopped all involvement in the municipality for some 
years at this point, apart from the evaluation itself, stake- 
holders largely self-organized and decided to work into 
the evening to define their response to the findings of 
the sustainability evaluation. 


The sustainability framework did not offer a perfect 
measurement tool, but served the role of a local systems 
thinking and learning tool 

The ongoing learning through the project phases of im- 
plementation and assessment took place despite some 
elements of the Sustainability Framework lacking mea- 
sures. For example, indicators of community capacity fo- 
cused on community-based organizations (WHCs) but 
failed to capture larger social processes and social capital 
formation, probably elements of equal if not greater im- 
port. The literature suggests that this remains a chal- 
lenge especially outside of research programs [38]. 

Some components of the Sustainability Framework are 
more amenable to standard, reliable measures than other 
components, assessed through softer methods. It is the 
resulting combined evidence which allowed actors of the 
system to engage in systems thinking and making sense 
of the data: 


e Proxy measures of health outcomes are well codified 
through both demographic and health surveys 
[27-29] and small population surveys [35], such 
as the KPC Survey used by Concern [36]. 

e The HICAP results were initially extremely useful in 
identifying structural weaknesses in the urban health 
institutional infrastructure. The heavy investment of 
the municipalities in refining and adapting the tool, 
while it did not allow for standardization, provided 
strong buy-in and critical reflection from partici- 
pants on their collective capacity, even if the benefit 
of the tool waned by 2009 (due to ceiling effects and 
lack of new information provided by the tool). 

e The consideration of viability and social-economic 
environment forced a wider perspective in analyzing 
how the external health intervention plays out in the 
development context. The tool required the project 
to consider inter-relationships between municipal 
authorities, the Ministries of Health, and civil 
society. The process assisted stakeholders in vetting 
risks to the health promotion efforts and give them 
higher priority in their efforts than they would have 
otherwise. For example, flood mitigation measures 
and female mobility for both CHVs and health 
referrals became central tenants of the community 
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mobilization strategy, even if they had not been ex- 
plicit mandates of the original project. 


The Sustainability Framework provided a tool and a 
method to engage stakeholders in learning evidence- 
based steps. Two features of the tool proved useful: 


e Being systematic is a content issue: the Sustainability 
Framework helped users to consider distinct 
dimensions of progress systematically, each with 
defined content elements and corresponding 
measures. The identification of the components of 
the model had strong face validity and remained 
meaningful to local stakeholders throughout the 
process. 

e Being systemic is a process issue: the Sustainability 
Framework considered a local system and, before 
focusing on measures of capacity and performance, 
sought to understand the relationships, both existing 
and those to be negotiated, between the members of 
the system. 


The process for fleshing out the content of the plan- 
ning or evaluation model is one which requires connec- 
tions and interplay between diverse and interdependent 
entities in the system (i.e., the WHCs and the MHD). In 
so doing, stakeholders also are encouraged to interact 
and, if using the iterative process, over time, build some 
common language, trust, and goals (social capital) [39]. 

A crucial step in this process was the development of 
a common vision by all stakeholders and the continuous 
reference to the shared mandate it created. A system of 
local actors possibly lacking coherence, common cur- 
rency, trust, and positive experience of joint achieve- 
ments will be limited in the vision it can frame. In 
Saidpur and Parabatipur, this process was facilitated 
greatly by Concern at the onset. By the 5-year post- 
project period, key elements of that vision were still 
shared and alive (the existence and importance of WHCs 
and volunteers, the need to support the most vulnerable 
citizens from the worst shocks). The fact that the local 
vision built on the realization of a national policy cer- 
tainly helped local actors to define and embrace it ini- 
tially; but by the time of the final study, it was 
maintained not by the will of a fairly absent central gov- 
ernment, but by its meaning to local actors. 


Ownership is inherently challenged by external 

assistance; early consideration of sustainability and a 
Systems approach to sustainability evaluation can 
mitigate this risk 

An inherent tension has existed between the concepts of 
external assistance and local ownership over the course 
of the last 70 years of development assistance. The 
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Sustainability Framework does not pretend to offer the 
solution to this quandary, but it does support a local sys- 
tem of actors exploring different visions and possibilities 
for a more successful pursuit of a common goal. It also 
helps external actors become change agents of and con- 
tributors to this system. It offers a way to reduce the dis- 
placement of ownership, which money naturally brings 
to resource-constrained environments. 

The initial focus on defining the system, bringing dif- 
ferent groups to the table, and trying to build a common 
vision and compatible scenarios are possibly the most 
important learning steps advanced by the Sustainability 
Framework, along with regular monitoring and reviews. 
Through iteration of measures, negotiation and decision 
making, the process required by the Sustainability 
Framework can help the local system adapt to successes, 
to new events, and shocks. Sustainability means that the 
system is able to conceive of, then realistically adapt and 
develop new ideas, thus transforming or evolving the 
scenario its members initially imagined. 

Saidpur and Parbatipur clearly reached a stage where 
new options were conceivable to them through an insti- 
tutional infrastructure which reached their most vulner- 
able members. The sustainability assessment identified 
choices, which municipalities had to make to seize on 
these options. 


Conclusions 

The challenge for Concern was to provide an ongoing 
evaluation process which would be evidence-based, 
allow effective implementation of interventions by mu- 
nicipal structures, and inform the social and health 
system actors about progress toward sustainable health 
achievements. 

The process of joint visioning, planning, implementation, 
monitoring, assessment, review, and decision-making pro- 
vided Saidpur and Parbatipur with a systematic, if perfect- 
ible, approach to do this. The answer made sense to the 
local actors from communities and municipalities, includ- 
ing health officials who participated in the sustainability 
evaluation. In the end, the Sustainability Framework played 
maybe its most important role as a tool for engagement of 
and negotiation between local stakeholders. It offered a 
guide to self-learning and decision-making with an 
evidence-based focus on objectives and tangible public 
good (health indicators, WHCs, and CHV activities). 

Social, political, and organizational systems have the 
particularity of being purposeful complex adaptive sys- 
tems® [40], which means that agents are endowed with 
some level of free will to define their own individual 
strategies within a system, based on information received 
about other agents’ behaviors. A systems approach, op- 
erationalized through the Sustainability Framework, re- 
duced the tension in balancing sustainability and equity. 
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As all key stakeholders were present and engaged in the 
framing of a vision inclusive of equitable concerns, the 
development of action plans and metrics of success had 
to include prioritization of the neediest. Furthermore, the 
constant reference to data, metrics of progress understood 
by all, referring back to a long-term vision repeated with 
constancy and visualized in assessment reports, allowed the 
construction of what Geyer and Rihani call a “societal 
framework” through which the value of the public good be- 
ing pursued is reinforced for all [41]. 

Development aid’s efforts at scale up and acceleration 
of achievements are known to create stress on country 
systems, regardless of good intentions. This makes the 
question of sustainability still enormously critical to the 
future of global health and global development [42,43]. 
Development projects too often deal with sustainability 
as a false promise or a utopia, with statements such as, 
“the project will ensure sustainability three years from 
now by...”; this does not lend itself to shared account- 
ability for progress on an authentic process worth the 
efforts of beneficiaries, country stakeholders, project de- 
signers, implementers, and donors. Commitment to sus- 
tainability requires us to approach more honestly and 
rigorously its unfortunate complexity. And as illustrated 
in the section “A systems understanding of sustainabil- 
ity,” we are still learning to recombine or create tools to 
effectively use “systems thinking” on complex issues 
such as ownership, scale, and sustainability [43-45]. We 
hope to have illustrated the value of one such approach. 

Finally, given the evidence for the challenging condi- 
tions under which sustainability can develop at local 
levels and the time this requires, national governments 
themselves, with or without foreign aid, will benefit from 
more methodical and systems-oriented planning evalu- 
ation methods to a complex but essential question. 


Endnotes 

*http://www.coregroup.org/about-us/history-of-core- 
group 

In a follow-on project in new municipalities, Concern 
added a module to the external baseline and final house- 
hold surveys, allowing comparison of health outcomes 
across proxy wealth quintiles [46]. 

“With the exception of a USAID-supported project 
promoting facility deliveries, during the life and after the 
end of the project. 

44 discussion of complex adaptive systems behaviors of 
both municipalities and project is available [47]. 
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The application of systems thinking in health: 
why use systems thinking? 


David H Peters 


Abstract 


people’s health. 


Background 

In the rapidly changing field of global health, it is hard to 
know whether the recent attention to systems thinking is 
just another fad, or something more durable that offers 
usable insights for understanding and action. Some see 
systems thinking as providing a powerful language to 
communicate and investigate complex issues, while others 
are confused by the sizable and amorphous body of theor- 
ies, methods, and tools involved. Time will tell, of course, 
but in the meantime, it is helpful to consider why we 
would use systems thinking in a field that already draws 
upon a rich collection of theories, methods, and tools 
from the health sciences, social sciences, engineering, 
mathematics, and other disciplines. 


From mental models to explicit ones 

At its core, systems thinking is an enterprise aimed at see- 
ing how things are connected to each other within some 
notion of a whole entity. We often make connections 
when conducting and interpreting research, or in our pro- 
fessional practice when we make an intervention with an 
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expectation of a result. Anytime we talk about how some 
event will turn out, whether the event is an epidemic, a 
war, or other social, biological, or physical process, we are 
invoking some mental model about how things fit to- 
gether. However, rather than relying on implicit models, 
with hidden assumptions and no clear link to data, sys- 
tems thinking deploys explicit models, with assumptions 
laid out that can be calibrated to data and repeated by 
others. The word system is derived from the Greek sunis- 
tdnai, meaning “to cause to stand together.” If we consider 
that a system is a perceived whole, made up of parts that 
interact toward a common purpose, we recognize that the 
ability to perceive, and the quality of that perception, is 
also part of what causes a system to stand together. Sys- 
tems thinking is intended to improve the quality of those 
perceptions of the whole, its parts, and the interactions 
within and between levels. 

Every interpretation of a research result involves a 
model, whether it is a physical model used for experimen- 
tation, a statistical model used to estimate the relation- 
ships between variables, or a conceptual model about how 
elements are connected. A model is simply a way we com- 
pactly represent and understand an object, phenomenon, 
or system. As much as research involves observation and 
experimentation, I would argue that good research is also 
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about building and using explicit models rather than im- 
plicit ones. The real question is not whether we should be 
using systems thinking, as broadly described here, but ra- 
ther, which of the many theories, methods, and tools cur- 
rently associated with the field of systems thinking are 
most useful in particular settings. 

For example, where individual people interact directly 
with one another (e.g., transmitting disease) while moving 
about in an explicit space such as a city, agent-based mod- 
eling [1,2] may be especially powerful. In modeling how 
different agencies within a large public health system 
interact, social network theory [3] could be more directly 
relevant. 


Origins 

Systems thinking has largely developed as a field of 
inquiry and practice in the 20" century, and has multiple 
origins in disciplines as varied as biology, anthropology, 
physics, psychology, mathematics, management, and com- 
puter science. The term is associated with a wide variety 
of scientists, including the biologist Ludwig von Berta- 
lanffy who developed General System Theory; psychiatrist 
Ross Ashby and anthropologist Gregory Bateson who pio- 
neered the field of cybernetics; Jay Forrester, a computer 
engineer who launched the field of systems dynamics; sci- 
entists at the Santa Fe Institute, such as Noble Laureates 
Murray Gell-Mann and Kenneth Arrow, who have helped 
define complex adaptive systems [4]; and a wide variety of 
management thinkers, including Russell Ackoff, a pioneer 
in operations research, and Peter Senge, who has popular- 
ized the learning organization. Much of the work in sys- 
tems thinking has involved bringing together scientists 
from many disciplinary traditions, in many cases allowing 
them to transfer methods from one discipline to another 
(inter-disciplinarity), or to work across and between dis- 
ciplinary boundaries, creating learning through a wide var- 
iety of stakeholders, including researchers and those 
affected by the research (trans-disciplinarity). 


Theories, methods, and tools 

If there is a jungle of terminology used to describe scien- 
tific endeavor, it gets even thicker in the area of systems 
thinking, perhaps because of its diverse heritage. Given 
the varied disciplines and trans-disciplinary traditions in- 
volved, it is easy to see why people often talk about 
broader “approaches”, “perspectives”, or “lenses” when ap- 
plying systems thinking. Systems thinking models and 
frameworks are sometimes grand and widely applicable, 
such as General System Theory, and at other times very 
specifically applied to particular phenomena, such as the 
theory on critical points in physics, which is used to ex- 
plain the point at which a material behaves as neither li- 
quid or gas (or solid). Systems thinking can involve a wide 
range of theories, which are rational sets of ideas or 
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principles intended to explain something. It is based on a 
wide variety of scientific methods used to investigate phe- 
nomena and acquire knowledge. It uses an even larger 
array of instruments or tools — the hardware and software 
used to conduct experiments, make observations, or col- 
lect and analyze data. The use of these terms is not con- 
sistent across or within scientific fields, including systems 
sciences, and the continuum from tool to method to the- 
ory and framework is often blurry. 

Rather than attempt to sort out semantic nuances be- 
tween these terms, the utility of systems thinking can be 
better appreciated by a brief look at some of its more com- 
monly used theories, methods, and tools (Table 1). The 
theories and methods in systems thinking are each de- 
signed to address complex problems. They are complex be- 
cause they involve multiple interacting agents, the context 
in which they operate keeps changing, because the manner 
in which things change do not conform to linear or simple 
patterns, or because elements within the system are able to 
learn new things, sometimes creating new patterns as they 
interact over time. Many of the challenges in global health 
are now recognized as complex problems where simple 
blueprint approaches have limited success [5,6]. 

Systems thinking tools have a wide variety of applica- 
tions. Some tools are intended as means of facilitating 
groups of people to have a common understanding about 
an issue to prompt further inquiry and action. For example, 
“systems archetypes” help teams to understand generic pat- 
terns of interaction that can be applicable to their “story” 
[24]. Rather than use the pre-existing templates of systems 
archetypes, causal loop diagrams (CLD) are created with- 
out a template, and involve drawing out people's under- 
standing of how elements of a problem are related to each 
other [19]. They usually begin as qualitative descriptions 
outlining how one thing causes another in either a positive 
or negative direction. Typically, feedback loops are identi- 
fied between the different elements. They can be reinfor- 
cing or positive feedback loops, where A produces more B 
which in turn produces more A, such as the vicious cycle 
of under-nutrition and infection. They can also be balan- 
cing or negative feedback loops, where a positive change in 
one leads to a push back in the opposite direction, such as 
when increasing body temperature produces sweating, 
which in turn cools down the body. In this supplement, a 
number of studies use CLDs that describe relationships be- 
tween different elements of a health system to explain phe- 
nomena such as dual practice of health workers in Uganda 
[25], provider payment systems in Ghana [26], and child- 
hood vaccination coverage in India [27]. 

The elements of a CLD might also be converted into a 
quantitative systems dynamics model by classifying the 
elements as “stocks”, “flows”, or “auxiliary” variables, and 
using equations to describe the relationships between in- 
dividual variables in one of many available systems 
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Table 1 Systems thinking theories, methods, 


Name 
Theories 


Catastrophe theory 


Cybernetics 


Chaos theory 


General systems theory 


Learning organizations 
theory 


Path dependency theories 


Punctuated equilibrium (in 
social theory) 


Methods 


Agent-based modeling 
(ABM) 


Network Analysis (or Social 
Network Analysis) 


Scenario planning 


Systems dynamics modeling 


Tools 


Causal loop diagrams (CLDs) 


Innovation (or change 
management) history 


Participatory Impact 
Pathways Analysis (PIPA) 


HEALTH RESEARCH 


POLICY AND SYSTEMS 


Purpose and description ho.. 


A theory in mathematics and geometry to study how small changes in parameters of a non- 
linear system can lead to sudden and large changes in behavior of a system. 


Historically used as a synonym for systems theory, it is a field of study of the communication 
and control of regulatory feedback in both living and non-living systems (e.g., organizations, 


machines). 


A field of study in mathematics with applications in a wide number of disciplines to explain a 
dynamic system and that is highly sensitive to the initial conditions, so that small changes in 
initial conditions produce wildly different results. The changes occur through fixed rules about 
changing relationships, and without randomness. 


Less of a theory than a way of finding a general theory to explain systems in all fields of 
science. It was not intended to be a single theory of systems, but more of a systematic inquiry 
into different domains of philosophy, science, and technology. 


A description of organizations that facilitate learning by its members and continuously 
transforms itself. Systems thinking approaches are the conceptual basis for understanding the 
organization in its environment, and provides a basis for other key characteristics, namely a 
process of learning (personal mastery), the challenging and building of mental models, and 
the development of a shared vision and team learning. 


Occurs in economics, social sciences, and physics, and refers to the explanations for why 
processes can have similar starting points yet lead to different outcomes, even if they follow 
the same rules, and outcomes are sensitive not only to initial conditions, but also to 
bifurcations and choices made along the way. 


Theory inspired from evolutionary biology [13] to explain long periods of stasis interrupted by 
rapid and radical change, particularly as applied to the evolution of policy change or conflict. 


ABMs are used to create a virtual representation of a complex system, modeling individual 
agents who interact with each other and the environment. Although the interactions are 
based on simple, pre-defined rules, in a complex system these simulations allow for the identi- 
fication of emergence and self-organization. 


Network analysis uses graphical methods to demonstrate relations between objects. 
Grounded in computer science, it has applications in social, biological, and physical sciences. 
Social network analysis involves application of network theory to social entities (e.g., people, 
groups, organizations), demonstrating nodes (individual actors within a network), and ties (the 
type of relationships) between the actors, and uses a range of tools for displaying the 
networks and analyzing the nature of the relationships. 


This is a strategic planning method that uses a series of tools to identify and analyze possible 
future events and alternative possible outcomes. These can involve quantitative projections 
and/or qualitative judgments about alternatives. The value lies more in learning from the 
planning process than the actual plans or scenarios. 


Not a single method, but an approach that uses a set of tools to understand the behavior of 
complex systems over time. The methods focus on the concepts of stocks and flows and 
feedback loops. They are designed to solve the problem of simultaneity (mutual causation) by 
being able to change variables over small periods of time while allowing for feedback and 
various interactions and delays. The common tools include causal loop diagrams and stock 
and flow diagrams. 


CLDs are a system dynamics tool that produces qualitative illustrations of mental models, 
focused on highlighting causality and feedback loops. Feedback loops can be either 
reinforcing or balancing, and CLDs can help to explain the role of such loops within a given 
system. CLDs are often developed in a participatory approach. The drawings can be further 
developed by categorizing the types of variables and quantifying the relationships between 
variables to form a stock and flow diagram. 


Innovation or change management history aims to generate knowledge about a system by 
compiling a systematic history of key events, intended and unintended outcomes, and 
measures taken to address emergent issues. It involves in-depth interviews with as many key 
stakeholders as possible to build an understanding of the performance of the system from a 
number of different points of view. 


PIPA is a workshop-based approach that combines impact pathway logic models and network 
mapping through a process involving stakeholder engagement. PIPA workshops aim to help 
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Table 1 Systems thinking theories, methods, and tools (Continued) 


participants to make their assumptions and underlying mental models about how projects 
run explicit and to reach consensus on how to achieve impact. 


Process mapping 


A set of tools, such as flow charts, to provide a pictorial representation of a sequence of 
actions and responses. Their use can be quite flexible, such as to make clear current processes, 


Damelio [22] 


as a basis for identifying bottlenecks or inefficient steps, or to produce an ideal map of how 


they would like them to be. 
Stock and flow diagrams 


Stock and flow diagrams are quantitative system dynamics tools used for illustrating a system Sterman [23] 


that can be used for model-based policy analysis in a simulated, dynamic environment. Stock 
and flow diagrams explicitly incorporate feedback to understand complex system behavior 


and capture non-linear dynamics. 
Systems archetypes 


Systems archetypes are a number of generic structures that describe common behaviors 
between the parts of a system. They provide templates to demonstrate different types of 


Kim [24] 


balancing and reinforcing feedback loops, which can be used by teams to come to a 
diagnosis about how a system is working, and particularly about how performance changes 


over time. 


dynamics software environments. In this supplement, 
Rwashana and colleagues use systems dynamics models to 
examine neonatal mortality in Uganda [28], while other 
authors use systems dynamics models to examine the ef- 
fects of policy interventions [29]. 

There are number of other tools that are used to map 
out events or how things are connected. Network mapping, 
social network analyses, and process mapping involve a 
range of tools to illustrate and analyze connections between 
people, organizations, or processes in both qualitative and 
quantitative ways. In this supplement, Malik et al. map out 
the network of actors involved in physician’s seeking advice 
in Pakistan [30]. The flow chart is one of the more com- 
mon tools used to draw a process or a system. Innovation 
history (or change management history) is used to compile 
a history of key events, outcomes, issues that have cropped 
up along the way, and measures taken to address problems. 
In this supplement, Zhang et al. [31] look back over the last 
35 years of the development of the medical system in rural 
China. Participatory Impact Pathways Analysis involves 
workshops and a combination of tools to clarify the logic of 
interventions and a mapping of the network [21]. It is 
intended to enhance understanding through participation 
with beneficiaries, implementers, and other stakeholders 
in a project. Several papers in this supplement use similar 
approaches for a variety of situations, including to build 
leadership capacity for health systems in South Africa 
[32], to develop sustainable physical rehabilitation pro- 
grams in Nepal and Somaliland [33], and to build sustain- 
able maternal and child preventive health services in 
Northern Bangladesh [34]. 

Agent based modeling takes advantage of a wide variety 
of theories, methods, and tools to build computer models 
that simulate the interaction of agents (e.g., individuals or 
organizations) to see how real world phenomena “grow 
and affect the system as a whole. The models involve mul- 
tiple individual agents that work at different scales, some 
decision-making rules (e.g., simple rules on how they 


reproduce, interact with others or pursue objectives), pro- 
cesses for adaptation, and a space in which the agents 
operate. 

In global health, we are concerned with both theory and 
practice, and are in need of models that match the com- 
plex conditions in which we work. A common thread of 
all these theories, methods, and tools is the idea that the 
behavior of systems is governed by common principles 
that can be discovered and expressed. They are all helpful 
in trying to conceptualize the systems in place. Some are 
more focused on ways to change the system to produce 
better outcomes. In using these theories, methods, and 
tools, we are reminded by the statistician George EP Box 
that “all models are wrong, but some are useful” |35]. It is 
to these uses that we now turn. 

In much of public health and medicine, we use research 
evidence on the efficacy of interventions to inform deci- 
sions with an expectation about their future effect. Some 
systems thinking methods and tools, such as scenario 
planning, can also be used to explicitly forecast future 
events. However, even then, such methods are intended to 
be used for identifying possible outcomes to provide in- 
sights on how to prepare for them rather than fixing on 
any particular outcome. 

In his landmark address on “Why Model?”, which pro- 
vided inspiration for this essay, Joshua Epstein identified 
16 reasons other than prediction on why to model [36]. 
Most of these reasons are applicable to systems thinking 
more broadly. Many of these specific reasons relate to be- 
ing able to explain how things work, and systems thinking 
is particularly useful to explaining how complex systems 
work. Many of models can be used for testing the viability 
of policy interventions in a safe and inexpensive way — 
agent based models, systems dynamics models, and sce- 
nario planning are particularly useful for these purposes. 
In this journal supplement, for example, Bishai et al. 
present a very simple systems dynamics model to illustrate 
the trade-offs and unintended consequences of policy 
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choices related to allocation to preventive and curative 
services [29]. 

Systems thinking approaches can also provide guidance 
on where to collect more data, or to raise new questions 
and hypotheses. The methods and tools help us to make 
explicit our assumptions, identify and test hypotheses, and 
calibrate our models against real data. One of the frustra- 
tions of health planners and researchers has been the as- 
piration that interventions shown to be effective at small 
scale or in a research setting cannot be simply replicated 
at large scale or to reach populations that are most vulner- 
able. Systems thinking methods and tools are increasingly 
being used to explain epidemics and to inform program- 
matic expansion efforts [5,6]. 

One of the more compelling reasons to use systems 
thinking approaches is to inspire a scientific habit of mind. 
Beyond the contributions of any particular theory, 
method, or tool, the practice of systems thinking can 
reinforce what Epstein calls a “militant ignorance”, or 
commitment to the principle that “I don’t know’ as a basis 
for expanding scientific knowledge. Systems thinking adds 
to the theories methods and tools we otherwise use in glo- 
bal health, and provides new opportunities to understand 
and continuously test and revise our understanding of the 
nature of things, including how to intervene to improve 
people's health. And for those who value thinking and 
doing in global health, that can only be a good thing. 
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